首页 > 系统相关 >阿里云k8s前端测试环境cpu和内存过低pod无法启动的问题

阿里云k8s前端测试环境cpu和内存过低pod无法启动的问题

时间:2023-01-05 16:22:59浏览次数:40  
标签:node talent verbose start test pod k8s cpu nuxt

镜像在本地可以正常启动,放到阿里云之后无法启动,只在容器启动的一瞬间

状态是oomkilled的然后无限重启容器,oomkilled状态只维持一会姑没有截图

 阿里云edas配置使用cpu和内存的资源,在开发更新代码之前项目一直稳定正常运行

 

 

 

 

#启动脚本
[root@deploy-nb-63 nuxt_talent_mobile]# cat run.sh 
#!/bin/sh
cd /nuxtjs && npm run start:test

 

阿里云ack检测到的报错

  Initialized: True

Ready: False
ContainersReady: False
PodScheduled: True
CrashLoopBackOFF

 

 #其中ack查看到的报错日志

 

 

 

#查看到报错文件
/nuxtjs # cat /root/.npm/_logs/2023-01-05T03_03_09_262Z-debug.log 0 info it worked if it ends with ok 1 verbose cli [ '/usr/local/bin/node', '/usr/local/bin/npm', 'run', 'start:test' ] 2 info using [email protected] 3 info using [email protected] 4 verbose run-script [ 'prestart:test', 'start:test', 'poststart:test' ] 5 info lifecycle [email protected]~prestart:test: [email protected] 6 info lifecycle [email protected]~start:test: [email protected] 7 verbose lifecycle [email protected]~start:test: unsafe-perm in lifecycle true 8 verbose lifecycle [email protected]~start:test: PATH: /usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/nuxtjs/node_modules/.bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 9 verbose lifecycle [email protected]~start:test: CWD: /nuxtjs 10 silly lifecycle [email protected]~start:test: Args: [ 10 silly lifecycle '-c', 10 silly lifecycle 'cross-env NODE_ENV=development ENV=development HOST=0.0.0.0 PORT=9700 node --max-old-space-size=4096 ./start.js' 10 silly lifecycle ] 11 silly lifecycle [email protected]~start:test: Returned: code: 1 signal: null 12 info lifecycle [email protected]~start:test: Failed to exec start:test script 13 verbose stack Error: [email protected] start:test: `cross-env NODE_ENV=development ENV=development HOST=0.0.0.0 PORT=9700 node --max-old-space-size=4096 ./start.js` 13 verbose stack Exit status 1 13 verbose stack at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:332:16) 13 verbose stack at EventEmitter.emit (events.js:400:28) 13 verbose stack at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14) 13 verbose stack at ChildProcess.emit (events.js:400:28) 13 verbose stack at maybeClose (internal/child_process.js:1058:16) 13 verbose stack at Process.ChildProcess._handle.onexit (internal/child_process.js:293:5) 14 verbose pkgid [email protected] 15 verbose cwd /nuxtjs 16 verbose Linux 3.10.0-1127.13.1.el7.x86_64 17 verbose argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "start:test" 18 verbose node v14.18.3 19 verbose npm v6.14.15 20 error code ELIFECYCLE 21 error errno 1 22 error [email protected] start:test: `cross-env NODE_ENV=development ENV=development HOST=0.0.0.0 PORT=9700 node --max-old-space-size=4096 ./start.js` 22 error Exit status 1 23 error Failed at the [email protected] start:test script. 23 error This is probably not a problem with npm. There is likely additional logging output above. 24 verbose exit [ 1, true ]

以上node的报错发送给开发没有排查出任何错误

 

 

#在本地测试服务器修改启动脚本然后手工启动正常,推送到阿里云手工测试
[root@deploy-nb-63 nuxt_talent_mobile]# cat run.sh 
#!/bin/sh
#cd /nuxtjs && npm run start:test
tail -F /nuxtjs/start.js

#手工启动node,非常正常

 

 

#在次测试在阿里云ack直接进入pod手工启动,进入node服务器进入容器内手工执行npm run start:test启动node

#根据ack上面查看到的名称和对应的节点ip在对应的node节点找出容器

 

 

 

#进入到node服务器 

 

#查找出所在的容器

 

 

 

 

 #进入到pod

 

#手工启动node

 

#其中的/root/.npm/_logs/2023-01-05T06_45_07_959Z-debug.log日志内容
/nuxtjs # cat /root/.npm/_logs/2023-01-05T06_45_07_959Z-debug.log

0 info it worked if it ends with ok
1 verbose cli [ '/usr/local/bin/node', '/usr/local/bin/npm', 'run', 'start:test' ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'prestart:test', 'start:test', 'poststart:test' ]
5 info lifecycle [email protected]~prestart:test: [email protected]
6 info lifecycle [email protected]~start:test: [email protected]
7 verbose lifecycle [email protected]~start:test: unsafe-perm in lifecycle true
8 verbose lifecycle [email protected]~start:test: PATH: /usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/nuxtjs/node_modules/.bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
9 verbose lifecycle [email protected]~start:test: CWD: /nuxtjs
10 silly lifecycle [email protected]~start:test: Args: [
10 silly lifecycle '-c',
10 silly lifecycle 'cross-env NODE_ENV=development ENV=development HOST=0.0.0.0 PORT=9700 node --max-old-space-size=4096 ./start.js'
10 silly lifecycle ]
11 silly lifecycle [email protected]~start:test: Returned: code: 1 signal: null
12 info lifecycle [email protected]~start:test: Failed to exec start:test script
13 verbose stack Error: [email protected] start:test: `cross-env NODE_ENV=development ENV=development HOST=0.0.0.0 PORT=9700 node --max-old-space-size=4096 ./start.js`
13 verbose stack Exit status 1
13 verbose stack at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:332:16)
13 verbose stack at EventEmitter.emit (events.js:400:28)
13 verbose stack at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
13 verbose stack at ChildProcess.emit (events.js:400:28)
13 verbose stack at maybeClose (internal/child_process.js:1058:16)
13 verbose stack at Process.ChildProcess._handle.onexit (internal/child_process.js:293:5)
14 verbose pkgid [email protected]
15 verbose cwd /nuxtjs
16 verbose Linux 3.10.0-1127.13.1.el7.x86_64
17 verbose argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "start:test"
18 verbose node v14.18.3
19 verbose npm v6.14.15
20 error code ELIFECYCLE
21 error errno 1
22 error [email protected] start:test: `cross-env NODE_ENV=development ENV=development HOST=0.0.0.0 PORT=9700 node --max-old-space-size=4096 ./start.js`
22 error Exit status 1
23 error Failed at the [email protected] start:test script.
23 error This is probably not a problem with npm. There is likely additional logging output above.
24 verbose exit [ 1, true ]

 

 #最后发现这样无法排查出错误,并且把日志发送给开发,开发也找不到原因,所以在本地测试服务器用docker stats查看容器启动的时候所占用的资源

 

 

 

 

 

 

 

 

 

发现启动过程中cpu资源最高去到了300%,内存最高去到了3g多

#启动后资源占用率下降,CPU维持在0.01核,内存维持在923.4MB

 

最终发现是开发代码没做优化所造成启动一瞬间资源占用过高,造成无法启动,只能调整阿里云k8s集群pod的内存和cpu资源限制

调整之后pod正常启动,但让开发优化代码

 

 

标签:node,talent,verbose,start,test,pod,k8s,cpu,nuxt
From: https://www.cnblogs.com/yeyouqing/p/17027912.html

相关文章