首页 > 其他分享 >解决ERROR oslo_messaging.rpc.server UnixHTTPConnectionPool(host=‘localhost‘, port=None) Read timed out

解决ERROR oslo_messaging.rpc.server UnixHTTPConnectionPool(host=‘localhost‘, port=None) Read timed out

时间:2023-10-03 10:03:40浏览次数:51  
标签:None Read oslo server rpc ERROR messaging 8468


zun.common.exception.ZunException: Unexpected error: UnixHTTPConnectionPool(host=‘localhost’, port=None): Read timed out. (read timeout=60)

2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server [req-6ff62c08-fd25-4df6-8a81-d144956cbbd9 7db25ffff6314194822f5ada52569de5 ce76b778ff0b453aa7b486f9934f694e default - -] Exception during message handling: zun.common.exception.ZunException: Unexpected error: UnixHTTPConnectionPool(host=‘localhost’, port=None): Read timed out. (read timeout=60)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py”, line 445, in _make_request
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server six.raise_from(e, None)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “”, line 3, in raise_from
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py”, line 440, in _make_request
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server httplib_response = conn.getresponse()
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/lib64/python3.6/http/client.py”, line 1346, in getresponse
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server response.begin()
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/lib64/python3.6/http/client.py”, line 307, in begin
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server version, status, reason = self._read_status()
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/lib64/python3.6/http/client.py”, line 268, in _read_status
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server line = str(self.fp.readline(_MAXLINE + 1), “iso-8859-1”)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/lib64/python3.6/socket.py”, line 586, in readinto
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server return self._sock.recv_into(b)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/eventlet/greenio/base.py”, line 371, in recv_into
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server return self._recv_loop(self.fd.recv_into, 0, buffer, nbytes, flags)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/eventlet/greenio/base.py”, line 359, in _recv_loop
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server self._read_trampoline()
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/eventlet/greenio/base.py”, line 331, in _read_trampoline
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server timeout_exc=socket_timeout(‘timed out’))
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/eventlet/greenio/base.py”, line 210, in _trampoline
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server mark_as_closed=self._mark_as_closed)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/eventlet/hubs/init.py”, line 159, in trampoline
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server return hub.switch()
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/eventlet/hubs/hub.py”, line 313, in switch
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server return self.greenlet.switch()
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server socket.timeout: timed out
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred:
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/requests/adapters.py”, line 449, in send
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server timeout=timeout
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py”, line 756, in urlopen
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py”, line 532, in increment
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server raise six.reraise(type(error), error, _stacktrace)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/urllib3/packages/six.py”, line 735, in reraise
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server raise value
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py”, line 706, in urlopen
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server chunked=chunked,
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py”, line 447, in _make_request
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py”, line 337, in _raise_timeout
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server self, url, “Read timed out. (read timeout=%s)” % timeout_value
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host=‘localhost’, port=None): Read timed out. (read timeout=60)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred:
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/zun/common/utils.py”, line 226, in decorated_function
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/zun/compute/manager.py”, line 577, in container_show
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server container = self.driver.show(context, container)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/zun/container/docker/driver.py”, line 616, in show
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server response = docker.inspect_container(container.container_id)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/docker/utils/decorators.py”, line 19, in wrapped
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server return f(self, resource_id, *args, **kwargs)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/docker/api/container.py”, line 774, in inspect_container
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server self._get(self._url(“/containers/{0}/json”, container)), True
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/docker/utils/decorators.py”, line 46, in inner
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server return f(self, *args, **kwargs)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/docker/api/client.py”, line 237, in _get
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server return self.get(url, **self._set_request_timeout(kwargs))
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/requests/sessions.py”, line 555, in get
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server return self.request(‘GET’, url, **kwargs)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/requests/sessions.py”, line 542, in request
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server resp = self.send(prep, **send_kwargs)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/requests/sessions.py”, line 655, in send
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server r = adapter.send(request, **kwargs)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/requests/adapters.py”, line 529, in send
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server raise ReadTimeout(e, request=request)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host=‘localhost’, port=None): Read timed out. (read timeout=60)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred:
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/oslo_messaging/rpc/server.py”, line 165, in _process_incoming
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py”, line 309, in dispatch
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py”, line 229, in _do_dispatch
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server File “/usr/local/lib/python3.6/site-packages/zun/common/utils.py”, line 232, in decorated_function
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server raise e
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server zun.common.exception.ZunException: Unexpected error: UnixHTTPConnectionPool(host=‘localhost’, port=None): Read timed out. (read timeout=60)
2023-09-13 17:26:08.649 8468 ERROR oslo_messaging.rpc.server


解决方法:

查看mq连接数量:

rabbitmqctl list_connections | wc -l


rabbitmqctl list_connections

如果具柄不够,修改相关具柄数量:

  1. 在 /etc/security/limits.conf 添加:
mysql hard nofile 65535
mysql soft nofile 65535
 
* soft nofile 655350
* hard nofile 655350
 
* soft nproc 655350
* hard nproc 655350
 
* soft memlock unlimited
* hard memlock unlimited
  1. 创建文件夹:mkdir -p /etc/systemd/system/mariadb.service.d/
    添加文件 vim /etc/systemd/system/mariadb.service.d/limits.conf , 内容如下:
[Service]
LimitNOFILE=65535
  1. 修改 vim /etc/my.cnf 添加 max_connections 项,如下:
[mysqld]
max_connections=30000
  1. 刷新配置:systemctl daemon-reload ;
  2. 重启服务:systemctl restart mariadb

2.Nova 计算节点发现

nova-manage cell_v2 discover_hosts --verbose

3.rabbitmq 句柄满了

  1. 编辑 vim /usr/lib/systemd/system/rabbitmq-server.service 文件,
    在 [Service] 下添加 LimitNOFILE=300000 ;如下:
[Service]
LimitNOFILE=300000
  1. 刷新配置:systemctl daemon-reload ;
  2. 重启服务:systemctl restart rabbitmq-server ;

在控制节点重新创建 keystone 认证用户:

source ~/keystonerc_admin

openstack user del zun
openstack user create --domain default --password zun zun
openstack role add --project services --user zun admin

openstack user del kuryr
openstack user create --domain default --password kuryr kuryr
openstack role add --project services --user kuryr admin

重启控制节点的zun服务:
systemctl restart zun-api zun-wsproxy

重启计算节点的zun服务:
systemctl restart kuryr-libnetwork
systemctl restart zun-compute


在 RabbitMQ 中使用 rabbitmqctl list_connections 命令查看TCP/IP连接的统计信息,命令格式:

rabbitmqctl list_connections [connectioninfoitem...]

注意,connectioninfoitem 参数用于指示哪些信息项会包含在结果集中,结果集的列顺序将匹配参数的顺序。connectioninfoitem 可选值如下:

pid:与连接相关的 Erlang 进程 ID

name:连接的名称。

port:服务器端口

host:返回反向 DNS 获取的服务器主机名称,或者 IP 地址,或者未启用。

peer_port:服务器对端端口。当一个客户端与服务器连接时,这个客户端的端口就是 peer_port。

peer_host:返回反向 DNS 获取的对端主机名称,或者 IP 地址,或者未启用。

ssl:是否启用 SSL

ssl_protocol:SSL 协议,如 tlsvl

ssl_key_exchange: SSL 密钥交换算法,如:rsa

ssl_cipher: SSL 加密算法,如:aes_256_cbc

ssl_hash: SSL 哈希算法,如:sha

peer_cert_subject:对端的 SSL 安全证书的主题,基于 RFC4514 的形式

peer_cert_issuer:对端 SSL 安全证书的发行者,基于 RFC4514 的形式

peer_cert_validity:对端 SSL 安全证书的有效期。

state:连接状态,包括 starting、tuning、opening、running、flow、blocking、blocked、closing 和 closed 这几种。

channels:该连接中的信道个数。

protocol:使用的 AMQP 协议的版本,当前是{0,9,1}或者{0,8,0}。注意,如果客户端请求的是 AMQP 0-9 的连接,RabbitMQ 也会将其视为 0-9-1。

auth_mechanism:使用的 SASL 认证机制,如 PLAIN、AMQPLAIN、EXTERNAL、RABBIT-CR-DEMO 等

user:与连接相关的用户名。

vhost:与连接相关的 vhost 的名称。

timeout:连接超时/协商的心跳间隔,单位为秒。

frame_max:最大传输帧的大小,单位为 B

channel_max:此连接上信道的最大数量。如果值 0,则表示无上限,但客户端一般会将 0 转变为 65535

client_properties:在建立连接期间由客户端发送的信息属性。

recv_oct:收到的字节数。

recv_cnt:收到的数据包个数。

send_oct:发送的字节数。

send_cnt:发送的数据包个数。

send_pend:发送队列大小。

connected_at:连接建立的时间戳。

示例

(1)如果没有指定 connectioninfoitem 信息,那么默认会显示 user、peer_host、peer_port 和 state 这几项信息。

D:\server\rabbitmq_server-3.9.11\sbin> rabbitmqctl list_connections
Listing connections ...
user    peer_host       peer_port       state
root    127.0.0.1       9365    running

(2)手动指定 connectioninfoitem 信息,例如:

D:\server\rabbitmq_server-3.9.11\sbin> rabbitmqctl list_connections user peer_host peer_port state
Listing connections ...
user    peer_host       peer_port       state
root    127.0.0.1       9365    running


标签:None,Read,oslo,server,rpc,ERROR,messaging,8468
From: https://blog.51cto.com/u_12763213/7689645

相关文章

  • 第8期ThreadX视频教程:应用实战,将裸机工程移植到RTOS的任务划分,驱动和应用层交互,中断DM
    视频教程汇总帖:https://www.armbbs.cn/forum.php?mod=viewthread&tid=110519 这个是我们初学RTOS面临的最直接问题,很多时候,简单的RTOS机制明白了,API也会调用了,就是添加到RTOS后,总感觉那里不对劲,怎么使用才是正确姿势。针对这些问题,本期视频教程,我们ThreadX内核教程穿插一期实......
  • cpp: read .dat file
     ///<summary>///打开DAT文件///</summary>voidoperatefile(){ chardata[100]; constchar*fname="afile.dat"; //打开文件. ofstreamoutfile; outfile.open(fname,ios::in); if(!outfile) { cout<<"文件不存在!"<......
  • adoc转换html+UPF低功耗仿真例子+python转换C代码+readmemh的@使用
    adoc转换htmladoc这种格式是很多riscv文档使用的格式,该格式可以生成pdf,生成html。生成html的好处是,选中和翻译方便,复制粘贴方便。首先是gem软件要安装,这个软件似乎是ruby相关的(RubyGemsisapackagemanagerfortheRubyprogramminglanguagethatprovidesastandardform......
  • pthread实现多线程矩阵乘法
    #include<pthread.h>#include<stdio.h>#include<windows.h>#include<iostream>usingnamespacestd;#pragmacomment(lib,"pthreadVC2.lib")#definerowCount1300#definemediumCount1500#definecolumnCount5000#definen_threa......
  • TypeError: unsupported operand type(s) for |: 'type' and 'NoneType' [duplicate]
      str|Nonesyntaxisonlysupportedin3.10orlater.UsefromtypingimportOptionalname:Optional[str]=NoneForcaseswheretherighthandsideisn'tNoneortherearemorethantwotypes,youcanuseUnionfromtypingimportUnionfoo:U......
  • redis 连接报错read error on connection解决
    在使用redis队列处理的时候,有时候队列过长会遇到超时的情况。原因分析:查看了下php.ini文件里面有个参数default_socket_timeout=60,就是这个配置导致redis过60秒会自动断开。这个配置是什么意思?default_socket_timeout是socket流的超时参数,即socket流从建立到传输再到关闭整个过......
  • pip安装PyVCF后有问题:ImportError: cannot import name 'Reader' from 'parser'
    目录问题解决问题Python通过pip安装PyVCF成功,但运行脚本时出现问题:File"/home/theo/anaconda3/lib/python3.9/site-packages/vcftoolz/vcftoolz.py",line19,in<module>importvcfFile"/home/theo/anaconda3/lib/python3.9/site-packages/vcf/__init__.py"......
  • 深入剖析ThreadLocal使用场景、实现原理、设计思想
    前言ThreadLocal可以用来存储线程的本地数据,做到线程数据的隔离ThreadLocal的使用不当可能会导致内存泄漏,排查内存泄漏的问题,不仅需要熟悉JVM、利用好各种分析工具还耗费人工如果能明白其原理并正确使用,就不会导致各种意外发生本文将从使用场景、实现原理、内存泄漏、设计思想......
  • Linux命令(78)之read
    linux命令之read1.read介绍linux命令read用来接收键盘或其它文件的输入,得到输入后,read命令将接收的数据放入到标准变量中。2.read用法read[参数][变量名称]read常用参数参数说明-p后面跟提示信息-e可以使用命令补全功能-n输入文本的长度-s输入字符时不再屏幕上显示,例如:logi......
  • 比较 opacity: 0、visibility: hidden、display: none
    结构display:none:会让元素完全从渲染树中消失,渲染的时候不占据任何空间,不能点击,visibility:hidden:不会让元素从渲染树消失,渲染元素继续占据空间,只是内容不可见,不能点击opacity:0:不会让元素从渲染树消失,渲染元素继续占据空间,只是内容不可见,可以点击继承display:none和op......