可能有人注意到dremio 管理界面的统计信息为N/A ，而且因为web 端进行了一些额外显示处理，造成一些疑惑
界面效果
接口返回数据信息
目前从官方代码来说，此显示是正常的，因为计算的是百分比，而且当系统负载比较低的时候，结果基本就为0
web 处理
NodeActivityView.js
- 参考代码
[port]: {
node: () => (node.get("port") !== -1 ? node.get("port") : "N/A"),
},
[cpu]: {
node: () =>
node.get("cpu") !== 0
? `${NumberFormatUtils.roundNumberField(node.get("cpu"))}%`
: "N/A",
},
[memory]: {
node: () =>
node.get("memory") !== 0
? `${NumberFormatUtils.roundNumberField(node.get("memory"))}%`
: "N/A", // todo: check comps for digits. and fix so no need for parseFloat
},
[version]: {
node: () => node.get("version") || "-",
},
后端处理代码
ExecutorServiceImpl 类
- 参考代码
public static CoordExecRPC.NodeStats getNodeStatsFromContext(SabotContext context) {
final ThreadsIterator threads = new ThreadsIterator(context, null);
final MemoryIterator memoryIterator = new MemoryIterator(context, null);
final WorkStats stats = context.getWorkStatsProvider().get();
final CoordinationProtos.NodeEndpoint ep = context.getEndpoint();
final double load = stats.getClusterLoad();
final int configuredMaxWidth = (int) context.getClusterResourceInformation().getAverageExecutorCores(context.getOptionManager());
final int actualMaxWidth = (int) Math.max(1, configuredMaxWidth * stats.getMaxWidthFactor());
// 默认为0，计算的是百分比
double memory = 0;
double cpu = 0;
// get cpu
while(threads.hasNext()) {
ThreadsIterator.ThreadSummary summary = (ThreadsIterator.ThreadSummary) threads.next();
double cpuTime = summary.cpu_time == null ? 0 : summary.cpu_time;
double numCores = summary.cores;
cpu += (cpuTime / numCores);
}
// get memory
if(memoryIterator.hasNext()) {
MemoryIterator.MemoryInfo memoryInfo = ((MemoryIterator.MemoryInfo) memoryIterator.next());
memory = memoryInfo.direct_current * 100.0 / memoryInfo.direct_max;
}
String ip = null;
try {
ip = InetAddress.getLocalHost().getHostAddress();
} catch (UnknownHostException e) {
// no op
}
return CoordExecRPC.NodeStats.newBuilder()
.setCpu(cpu)
.setMemory(memory)
.setVersion(DremioVersionInfo.getVersion())
.setPort(ep.getFabricPort())
.setName(ep.getAddress())
.setIp(ip)
.setStatus("green")
.setLoad(load)
.setConfiguredMaxWidth(configuredMaxWidth)
.setActualMaxWith(actualMaxWidth)
.setCurrent(false)
.build();
}
- 接口port 为-1 的问题
如下，因为userpoprt 就是为-1（使用随机端口），fabricport 是确定的
接口返回数据的处理
Nodes 类
// 获取是是userport 就是-1，所以界面显示就就是N/A 了
public static NodeInfo fromEndpoint(CoordinationProtos.NodeEndpoint endpoint) {
final boolean master = endpoint.getRoles().getMaster();
final boolean coord = endpoint.getRoles().getSqlQuery();
final boolean exec = endpoint.getRoles().getJavaExecutor();
boolean isCompatible = isCompatibleVersion(endpoint.getDremioVersion());
return new NodeInfo(
endpoint.getAddress(),
endpoint.getAddress(),
endpoint.getAddress(),
endpoint.getUserPort(),
0d,
0d,
"green",
master,
coord,
exec,
isCompatible,
endpoint.getNodeTag(),
endpoint.getDremioVersion(),
endpoint.getStartTime(),
isCompatible ? NodeDetails.NONE.toMessage(null) : NodeDetails.INVALID_VERSION.toMessage(endpoint.getDremioVersion())
);
}
说明
dremio 对于一些信息缺少说明，结合源码查看是比较好的
参考资料
dac/ui/src/pages/AdminPage/subpages/NodeActivity/NodeActivityView.js
dac/backend/src/main/java/com/dremio/dac/resource/SystemResource.java
services/executorservice/src/main/java/com/dremio/service/executor/ExecutorServiceClient.java
sabot/kernel/src/main/java/com/dremio/exec/service/executor/ExecutorServiceProductClient.java
sabot/kernel/src/main/java/com/dremio/sabot/rpc/CoordExecService.java
sabot/kernel/src/main/java/com/dremio/exec/service/executor/ExecutorServiceImpl.java
sabot/kernel/src/main/java/com/dremio/exec/store/sys/ThreadsIterator.java
sabot/kernel/src/main/java/com/dremio/exec/store/sys/MemoryIterator.java
common/legacy/src/main/java/com/dremio/common/VM.java
sabot/kernel/src/main/java/com/dremio/exec/server/ContextService.java
dac/backend/src/main/java/com/dremio/dac/model/system/Nodes.java