docker swarm错误: Error response from daemon: rpc error: code = 4 desc = context deadline exceeded
docker swarm: Error response from daemon: rpc error: code = 4 desc = context deadline exceeded
碰到一个问题,记录在此。
错误内容:
所有docker swarm/service命令都不能执行,返回如下错误:
包括 $ docker node ls
等swarm相关的命令:
$ docker swarm join-token manager
Error response from daemon: rpc error: code = 4 desc = context deadline exceeded
但是docker命令都正常, 例如docker ps/docker images等等,只是docker swarm的命令无法执行。
原因分析:
查看docker的log
$ sudo tail -f /var/log/messages | grep docker
注意不同的Linux发行版本,docker log位置不一致;上述是Redhat的办法:
dockerd: time="2019-04-18T11:58:16.371148755+08:00" level=debug msg="miss notification: dest IP 10.0.0.4, dest MAC "
dockerd: time="2019-04-18T11:58:16.373118682+08:00" level=debug msg="miss notification: dest IP 10.0.0.8, dest MAC "
dockerd: time="2019-04-18T11:58:16.780678187+08:00" level=error msg="agent: session failed" error="session initiation timed out" module="node/agent" node.id=271z2mmy2mqqxe2msysm72q2s
dockerd: time="2019-04-18T11:58:16.780713226+08:00" level=debug msg="agent: rebuild session" module="node/agent" node.id=271z2mmy2mqqxe2msysm72q2s
dockerd: time="2019-04-18T11:58:17.371118179+08:00" level=debug msg="miss notification: dest IP 10.0.0.6, dest MAC "
dockerd: time="2019-04-18T11:58:17.373148037+08:00" level=debug msg="miss notification: dest IP 10.0.0.4, dest MAC "
dockerd: time="2019-04-18T11:58:18.371017818+08:00" level=debug msg="miss notification: dest IP 10.0.0.8, dest MAC "
dockerd: time="2019-04-18T11:58:18.373146453+08:00" level=debug msg="miss notification: dest IP 10.0.0.6, dest MAC "
dockerd: time="2019-04-18T11:58:18.643814757+08:00" level=info msg="3a81dc09ea33b88e is starting a new election at term 812" module=raft node.id=271z2mmy2mqqxe2msysm72q2s
dockerd: time="2019-04-18T11:58:18.643848592+08:00" level=info msg="3a81dc09ea33b88e became candidate at term 813" module=raft node.id=271z2mmy2mqqxe2msysm72q2s
dockerd: time="2019-04-18T11:58:18.643865419+08:00" level=info msg="3a81dc09ea33b88e received MsgVoteResp from 3a81dc09ea33b88e at term 813" module=raft node.id=271z2mmy2mqqxe2msysm72q2s
dockerd: time="2019-04-18T11:58:18.643899515+08:00" level=info msg="3a81dc09ea33b88e [logterm: 2, index: 141] sent MsgVote request to 8be3f17aae7108e at term 813" module=raft node.id=271z2mmy2mqqxe2msysm72q2s
dockerd: time="2019-04-18T11:58:18.645083884+08:00" level=debug msg="failed to send message MsgVote" error="rpc error: code = 13 desc = connection error: desc = \"transport: x509: certificate has expired or is not yet valid\"" peer_id=8be3f17aae7108e
dockerd: time="2019-04-18T11:58:19.371110322+08:00" level=debug msg="miss notification: dest IP 10.0.0.8, dest MAC "
dockerd: time="2019-04-18T11:58:19.371435727+08:00" level=debug msg="miss notification: dest IP 10.0.0.4, dest MAC "
说明swarm cluster环境通信遇到了问题。
解决办法:
重新初始化cluster环境即可:
$ docker swarm init --force-new-cluster
然后就一切正常了。