About Zeek¶ 关于zeek ¶
What Is Zeek?¶ 什么是zeek？ ¶
Zeek is a passive, open-source network traffic analyzer. Many operators use Zeek as a network security monitor (NSM) to support investigations of suspicious or malicious activity. Zeek also supports a wide range of trafficanalysis tasks beyond the security domain, including performance measurement and troubleshooting.
Zeek 是一款被动的开源网络流量分析器。许多运营商使用 Zeek 作为网络安全监视器 (NSM) 来支持对可疑或恶意活动的调查。 Zeek还支持安全域之外的广泛流量分析任务，包括性能测量和故障排除。
The first benefit a new user derives from Zeek is the extensive set of logs describing network activity. These logs include not only a comprehensive record of every connection seen on the wire, but also application-layer transcripts. These include all HTTP sessions with their requested URIs, key headers, MIME types, and server responses; DNS requests with replies; SSL certificates; key content of SMTP sessions; and much more. By default, Zeek writes all this information into well-structured tab-separated or JSON log files suitable for post-processing with external software. Users can also choose to have external databases or SIEM products consume, store, process, and present the data for querying.
新用户从 Zeek 获得的第一个好处是描述网络活动的大量日志。这些日志不仅包括线路上看到的每个连接的全面记录，还包括应用程序层的记录。其中包括所有 HTTP会话及其请求的 URI、密钥标头、MIME 类型和服务器响应；带回复的 DNS 请求； SSL 证书；SMTP会话的关键内容；以及更多。默认情况下，Zeek 将所有这些信息写入结构良好的制表符分隔或 JSON日志文件，适合使用外部软件进行后处理。用户还可以选择让外部数据库或 SIEM 产品使用、存储、处理和呈现数据以供查询。
In addition to the logs, Zeek comes with built-in functionality for a range ofanalysis and detection tasks, including extracting files from HTTP sessions,detecting malware by interfacing to external registries, reporting vulnerableversions of software seen on the network, identifying popular webapplications, detecting SSH brute-forcing, validating SSL certificate chains,and much more.
除了日志之外，Zeek 还具有用于一系列分析和检测任务的内置功能，包括从 HTTP会话中提取文件、通过连接到外部注册表来检测恶意软件、报告网络上出现的软件的易受攻击版本、识别流行的网络应用程序、检测 SSH 暴力破解、验证 SSL证书链等等。
In addition to shipping such powerful functionality “out of the box,” Zeek isa fully customizable and extensible platform for traffic analysis. Zeekprovides users a domain-specific, Turing-complete scripting language forexpressing arbitrary analysis tasks. Think of the Zeek language as a “domain-specific Python” (or Perl): just like Python, the system comes with a largeset of pre-built functionality (the “standard library”), yet users can alsoput Zeek to use in novel ways by writing custom code. Indeed, all of Zeek’sdefault analyses, including logging, are done via scripts; no specificanalysis is hard-coded into the core of the system.
除了“开箱即用”的强大功能之外，Zeek 还是一个完全可定制和可扩展的流量分析平台。 Zeek为用户提供了一种特定于领域的、图灵完备的脚本语言，用于表达任意分析任务。将 Zeek 语言视为“特定领域的 Python”（或 Perl）：就像Python 一样，系统附带大量预构建的功能（“标准库”），但用户也可以使用 Zeek通过编写自定义代码以新颖的方式。事实上，Zeek的所有默认分析（包括日志记录）都是通过脚本完成的；没有具体的分析被硬编码到系统的核心中。
Zeek runs on commodity hardware and hence provides a low-cost alternative toexpensive proprietary solutions. In many ways Zeek exceeds the capabilities ofother network monitoring tools, which typically remain limited to a small setof hard-coded analysis tasks. Zeek is not a classic signature-based intrusiondetection system (IDS); while it supports such standard functionality as well,Zeek’s scripting language facilitates a much broader spectrum of verydifferent approaches to finding malicious activity. These include semanticmisuse detection, anomaly detection, and behavioral analysis.
Zeek 在商用硬件上运行，因此为昂贵的专有解决方案提供了低成本替代方案。在许多方面，Zeek都超越了其他网络监控工具的功能，这些工具通常仅限于一小组硬编码的分析任务。 Zeek 不是一个经典的基于签名的入侵检测系统(IDS)；虽然它也支持此类标准功能，但 Zeek 的脚本语言促进了更广泛的不同方法来查找恶意活动。其中包括语义误用检测、异常检测和行为分析。
A large variety of sites deploy Zeek to protect their infrastructure,including many universities, research labs, supercomputing centers, open-science communities, major corporations, and government agencies. Zeekspecifically targets high-speed, high-volume network monitoring, and anincreasing number of sites are now using the system to monitor their 10GEnetworks, with some already moving on to 100GE links.
许多站点都部署了 Zeek 来保护其基础设施，包括许多大学、研究实验室、超级计算中心、开放科学社区、大型公司和政府机构。 Zeek专门针对高速、大容量网络监控，越来越多的站点现在使用该系统来监控其 10GE 网络，其中一些站点已经转向 100GE 链路。
Zeek accommodates high-performance settings by supporting scalable load-balancing. Large sites typically run “Zeek Clusters” in which a high-speedfront end load balancer distributes the traffic across an appropriate numberof back end PCs, all running dedicated Zeek instances on their individualtraffic slices. A central manager system coordinates the process,synchronizing state across the back ends and providing the operators with acentral management interface for configuration and access to aggregated logs.Zeek’s integrated management framework, ZeekControl, supports such clustersetups out-of-the-box.
Zeek 通过支持可扩展的负载平衡来适应高性能设置。大型站点通常运行“Zeek 集群”，其中高速前端负载均衡器将流量分配到适当数量的后端 PC，所有后端PC 都在各自的流量切片上运行专用的 Zeek 实例。中央管理器系统协调流程，同步后端状态，并为操作员提供中央管理界面，用于配置和访问聚合日志。 Zeek的集成管理框架 ZeekControl 支持此类开箱即用的集群设置。
Zeek’s cluster features support single-system and multi-system setups. That’spart of Zeek’s scalability advantages. For example, administrators can scaleZeek within one system for as long as possible, and then transparently addmore systems when necessary.
Zeek 的集群功能支持单系统和多系统设置。这是 Zeek 可扩展性优势的一部分。例如，管理员可以在一个系统内尽可能长时间地扩展Zeek，然后在必要时透明地添加更多系统。
In brief, Zeek is optimized for interpreting network traffic and generatinglogs based on that traffic. It is not optimized for byte matching, and usersseeking signature detection approaches would be better served by tryingintrusion detection systems such as Suricata. Zeek is also not a protocolanalyzer in the sense of Wireshark, seeking to depict every element of networktraffic at the frame level, or a system for storing traffic in packet capture(PCAP) form. Rather, Zeek sits at the “happy medium” representing compact yethigh fidelity network logs, generating better understanding of network trafficand usage.
简而言之，Zeek 针对解释网络流量并根据该流量生成日志进行了优化。它没有针对字节匹配进行优化，寻求签名检测方法的用户可以通过尝试 Suricata等入侵检测系统得到更好的服务。 Zeek 也不是 Wireshark 意义上的协议分析器，它试图在帧级别描述网络流量的每个元素，也不是用于以数据包捕获(PCAP) 形式存储流量的系统。相反，Zeek 处于“快乐媒介”，代表紧凑但高保真度的网络日志，可以更好地理解网络流量和使用情况。
Why Zeek?¶ 为什么是zeek？ ¶
Zeek offers many advantages for security and network teams who want to betterunderstand how their infrastructure is being used.
Zeek 为希望更好地了解其基础设施使用方式的安全和网络团队提供了许多优势。
Security teams generally depend upon four sorts of data sources when trying todetect and respond to suspicious and malicious activity. These include thirdparty sources such as law enforcement, peers, and commercial or nonprofitthreat intelligence organizations; network data ; infrastructure andapplication data , including logs from cloud environments; and endpointdata. Zeek is primarily a platform for collecting and analyzing the secondform of data – network data. All four are important elements of any securityteam’s program, however.
安全团队在尝试检测和响应可疑和恶意活动时通常依赖四种数据源。其中包括第三方来源，例如执法机构、同行以及商业或非营利威胁情报组织；网络数据；基础设施和应用程序数据，包括来自云环境的日志；和端点数据。Zeek主要是一个收集和分析第二种数据形式——网络数据的平台。然而，这四个因素都是任何安全团队计划的重要组成部分。
When looking at data derived from the network, there are four types of dataavailable to analysts. As defined by the network security monitoringparadigm,these four data types are full content , transaction data , extractedcontent , and alert data. Using these data types, one can record traffic,summarize traffic, extract traffic (or perhaps more accurately, extractcontent in the form of files), and judge traffic, respectively.
在查看来自网络的数据时，分析师可以使用四种类型的数据。根据网络安全监控范式的定义，这四种数据类型是完整内容、交易数据、提取内容和警报数据。使用这些数据类型，可以分别记录流量、汇总流量、提取流量（或者更准确地说，提取文件形式的内容）和判断流量。
It’s critical to collect and analyze the four types of network securitymonitoring data. The question becomes one of determining the best way toaccomplish this goal. Thankfully, Zeek as a NSM platform enables collection ofat least two, and in some ways three, of these data forms, namely transactiondata, extracted content, and alert data.
收集和分析四类网络安全监控数据至关重要。问题变成了确定实现这一目标的最佳方法。值得庆幸的是，Zeek 作为 NSM平台，可以收集至少两种（甚至在某些方面是三种）这些数据形式，即交易数据、提取的内容和警报数据。
Zeek is best known for its transaction data. By default, when run and told towatch a network interface, Zeek will generate a collection of compact, high-fidelity, richly-annotated set of transaction logs. These logs describe theprotocols and activity seen on the wire, in a judgement-free, policy-neutralmanner. This documentation will spend a considerable amount of time describingthe most common Zeek log files such that readers will become comfortable withthe format and learn to apply them to their environment.
Zeek 以其交易数据而闻名。默认情况下，当运行并被告知监视网络接口时，Zeek将生成一组紧凑、高保真、注释丰富的事务日志集。这些日志以不加判断、政策中立的方式描述了网络上看到的协议和活动。本文档将花费大量时间描述最常见的 Zeek日志文件，以便读者熟悉该格式并学习将它们应用到自己的环境中。
Zeek can also easily carve files from network traffic, thanks to its fileextraction capabilities. Analysts can then send those files to executionsandboxes or other file examination tools for additional investigation. Zeekhas some capability to perform classical byte-centric intrusion detection, butthat job is best suited for packages like the open source Snort or Suricataengines. Zeek has other capabilities however that are capable of providingjudgements in the form of alerts, through its notice mechanism.
凭借其文件提取功能，Zeek 还可以轻松地从网络流量中提取文件。然后，分析人员可以将这些文件发送到执行沙箱或其他文件检查工具以进行进一步调查。 Zeek具有执行经典的以字节为中心的入侵检测的能力，但这项工作最适合开源 Snort 或 Suricata 引擎等软件包。然而，Zeek还有其他功能，能够通过其通知机制以警报的形式提供判断。
Zeek is not optimized for writing traffic to disk in the spirit of a fullcontent data collection, and that task is best handled by software written tofulfill that requirement.
Zeek 并未本着完整内容数据收集的精神将流量写入磁盘进行优化，并且该任务最好由为满足该要求而编写的软件来处理。
Beyond the forms of network data that Zeek can natively collect and generate,Zeek has advantages that appeared in the What Is Zeek? section. These includeits built-in functionality for a range of analysis and detection tasks, andits status as a fully customizable and extensible platform for trafficanalysis. Zeek is also attractive because of its ability to run on commodityhardware, giving users of all types the ability to at least try Zeek in a low-cost manner.
除了 Zeek 本身可以收集和生成的网络数据形式之外，Zeek 还具有“什么是Zeek？”中出现的优势。部分。其中包括其用于一系列分析和检测任务的内置功能，以及其作为完全可定制和可扩展的流量分析平台的地位。 Zeek的吸引力还在于它能够在商用硬件上运行，使所有类型的用户至少能够以低成本方式尝试 Zeek。
History¶ 历史 ¶
Zeek has a rich history stretching back to the 1990s. VernPaxson designed and implemented the initialversion in 1995 as a researcher at the Lawrence Berkeley National Laboratory(LBNL). The original software was called “Bro,” as an“Orwellian reminder that monitoring comes hand in hand with the potential forprivacy violations”.
Zeek 的悠久历史可以追溯到 20 世纪 90 年代。 Vern Paxson 于 1995 年作为劳伦斯伯克利国家实验室 (LBNL)的研究员设计并实现了最初的版本。最初的软件被称为“Bro”，作为“奥威尔式的提醒，监控与潜在的隐私侵犯密切相关”。
LBNL first deployed Zeek in 1996, and the USENIX Security Symposium publishedVern’s original paper on Zeek in 1998, and awarded it the Best Paper Awardthat year He published a refined version of the paper in 1999 as Bro: ASystem for Detecting Network Intruders in Real-Time.
LBNL 于 1996 年首次部署了 Zeek，USENIX 安全研讨会于 1998 年发表了 Vern 关于 Zeek的原始论文，并授予当年最佳论文奖。他于 1999 年发表了该论文的精炼版本，名为《Bro: A System for Detecting NetworkIntruders》即时的。
In 2003, the National Science Foundation (NSF) begansupporting research and advanced development on Bro at the InternationalComputer Science Institute (ICSI). (Vern stillleads the ICSI Networking and Security group.)
2003 年，美国国家科学基金会 (NSF) 开始支持国际计算机科学研究所 (ICSI) 对 Bro 的研究和高级开发。 （Vern 仍然领导着 ICSI网络和安全小组。）
Over the years, a growing team of ICSI researchers and students kept addingnovel functions to Zeek, while LBNL continued its support with funding fromthe Department of Energy (DOE). Much of Zeek’scapabilities originate in academic research projects, with results oftenpublished at top-tier conferences. A key to Zeek’s success was the project’sability to bridge the gap between academia and operations. This relationshiphelped ground research on Zeek in real-world challenges.
多年来，越来越多的 ICSI 研究人员和学生团队不断为 Zeek 添加新功能，而劳伦斯伯克利国家实验室 (LBNL) 则在能源部 (DOE)的资助下继续提供支持。 Zeek 的大部分能力源自学术研究项目，其研究成果经常在顶级会议上发表。 Zeek成功的关键是该项目能够弥合学术界和运营之间的差距。这种关系有助于 Zeek 在现实世界的挑战中进行研究。
With a growing operational user community, the research-centric developmentmodel eventually became a bottleneck to the system’s evolution. Researchgrants did not support the more mundane parts of software development andmaintenance. However, those elements were crucial for the end-user experience.As a result, deploying Zeek required overcoming a steep learning curve.
随着运营用户社区的不断壮大，以研究为中心的开发模式最终成为系统演进的瓶颈。研究经费不支持软件开发和维护中更平凡的部分。然而，这些元素对于最终用户体验至关重要。因此，部署Zeek 需要克服陡峭的学习曲线。
In 2010, NSF sought to address this challenge by awarding ICSI a grant fromits Software Development for Cyberinfrastructure fund. The National Centerfor Supercomputing Applications (NCSA) joinedthe team as a core partner, and the Zeek project began to overhaul many of theuser-visible parts of the system for the 2.0 release in 2012.
2010 年，NSF 试图通过其网络基础设施软件开发基金向 ICSI提供资助来应对这一挑战。国家超级计算应用中心（NCSA）作为核心合作伙伴加入该团队，Zeek 项目开始于 2012 年发布 2.0版本，彻底改造系统的许多用户可见部分。
After Zeek 2.0, the project enjoyed tremendous growth in new deploymentsacross a diverse range of settings, and the ongoing collaboration between ICSI(co-PI Robin Sommer) and NCSA (co-PI Adam Slagell) brought a number ofimportant features. In 2012, Zeek added native IPv6 support, long before manyenterprise networking monitoring tools. In 2013, NSF renewed its support witha second grant that established the Bro Center of Expertise at ICSI and NCSA,promoting Zeek as a comprehensive, low-cost security capability for researchand education communities. To facilitate both debugging and education,try.zeek.org (formerly try.bro.org) was launched in2014. This provided an interactive way for users to test a script with theirown packet captures against a variety of Zeek versions and easily share samplecode with others. For Zeek clusters and external communication, the Brokercommunication framework was added. Last, but not least, the Zeek packagemanager was created in 2016, funded by an additional grant from the MozillaFoundation. Zeek 2.0 之后，该项目在各种环境中的新部署中取得了巨大的增长，并且 ICSI（联合 PI Robin Sommer）和 NCSA（联合 PIAdam Slagell）之间的持续合作带来了许多重要功能。 2012 年，Zeek 增加了原生 IPv6 支持，早于许多企业网络监控工具。 2013年，NSF 再次提供支持，提供了第二笔资助，在 ICSI 和 NCSA 建立了 Bro 专业知识中心，将 Zeek推广为研究和教育社区的全面、低成本安全功能。为了促进调试和教育，try.zeek.org（以前称为 try.bro.org）于 2014年推出。这为用户提供了一种交互式方式，可以使用自己的数据包捕获针对各种 Zeek版本来测试脚本，并轻松共享与其他人一起使用示例代码。对于Zeek集群和外部通信，增加了Broker通信框架。最后但并非最不重要的一点是，Zeek 包管理器于2016 年创建，由 Mozilla 基金会提供额外资助。
In the fall of 2018, the project leadership team decided to change the name ofthe software from Bro to Zeek. The leadership team desired a name that betterreflected the values of the community while avoiding the negative connotationsof so-called “bro culture” outside the computing world. The project releasedversion 3.0 in the fall of 2019, the first release bearing the name Zeek. Theyear 2020 saw a renewed focus on community and growing the Zeek community,with increased interaction via social media, webinars, Slack channels, andrelated outreach efforts.
2018年秋天，项目领导团队决定将软件名称从Bro更改为Zeek。领导团队希望一个名称能够更好地反映社区的价值观，同时避免计算机世界之外所谓的“兄弟文化”的负面含义。该项目于2019 年秋季发布了 3.0 版本，第一个版本名为 Zeek。 2020 年，人们重新关注社区并不断发展 Zeek社区，通过社交媒体、网络研讨会、Slack 渠道和相关外展工作增加了互动。
For a history of the project from 1995 to 2015, see Vern Paxson’s talk fromBroCon 2015, Reflecting on Twenty Years ofBro.
有关该项目从 1995 年到 2015 年的历史，请参阅 Vern Paxson 在 BroCon 2015 上的演讲，Reflecting onTwenty of Bro.
For background on the decision to rename Bro to Zeek, see Vern Paxson’s talkfrom BroCon 2018, Renaming Bro.
有关将 Bro 重命名为 Zeek 的决定的背景信息，请参阅 Vern Paxson 在 BroCon 2018 上的演讲，重命名 Bro。
Architecture¶ 架构 ¶
At a very high level, Zeek is architecturally layered into two majorcomponents. Its event engine (or core ) reduces the incoming packet streaminto a series of higher-level events. These events reflect network activityin policy-neutral terms, i.e., they describe what has been seen, but not_why_ , or whether it is significant.
在非常高的层面上，Zeek在架构上分为两个主要组件。其事件引擎（或核心）将传入的数据包流简化为一系列更高级别的事件。这些事件以政策中立的方式反映了网络活动，即它们描述了所看到的内容，但没有描述原因或是否重要。
For example, every HTTP request on the wire turns into a correspondinghttp_request event that carries with it the involved IPaddresses and ports, the URI being requested, and the HTTP version in use. Theevent however does not convey any further interpretation , such as whetherthat URI corresponds to a known malware site.
例如，线路上的每个 HTTP 请求都会变成相应的 http_request 事件，其中包含所涉及的 IP 地址和端口、所请求的 URI 以及所使用的HTTP 版本。然而，该事件不会传达任何进一步的解释，例如该 URI 是否对应于已知的恶意软件站点。
The event engine component comprises a number of subcomponents, including inparticular the packet processing pipeline consisting of: input sources, packetanalysis, session analysis, and file analysis. Input sources ingest incomingnetwork traffic from network interfaces. Packet analysis processes lower-levelprotocols, starting all the way down at the link layer. Session analysishandles application-layer protocols, such as HTTP, FTP, etc. File analysisdissects the content of files transferred over sessions. The event engineprovides a plugin architecture for adding any of these from outside of thecore Zeek code base, allowing to expand Zeek’s capabilities as needed.
事件引擎组件包括多个子组件，特别包括数据包处理管道，其组成为：输入源、数据包分析、会话分析和文件分析。输入源从网络接口摄取传入网络流量。数据包分析处理较低层的协议，从链路层一直向下开始。会话分析处理应用层协议，例如HTTP、FTP 等。文件分析剖析通过会话传输的文件内容。事件引擎提供了一个插件架构，用于从核心 Zeek 代码库外部添加任何这些内容，从而允许根据需要扩展Zeek 的功能。
Semantics related to the events are derived by Zeek’s second main component,the script interpreter , which executes a set of event handlers written inZeek’s custom scripting language. These scripts can express a site’s securitypolicy, such as what actions to take when the monitor detects different typesof activity.
与事件相关的语义由 Zeek 的第二个主要组件（脚本解释器）导出，它执行一组用 Zeek的自定义脚本语言编写的事件处理程序。这些脚本可以表达站点的安全策略，例如当监视器检测到不同类型的活动时要采取哪些操作。
More generally scripts can derive any desired properties and statistics fromthe input traffic. In fact, all of Zeek’s default output comes from scriptsincluded in the distribution. Zeek’s language comes with extensive domain-specific types and support functionality. Crucially, Zeek’s language allowsscripts to maintain state over time, enabling them to track and correlate theevolution of what they observe across connection and host boundaries. Zeekscripts can generate real-time alerts and also execute arbitrary externalprograms on demand. One might use this functionality to trigger an activeresponse to an attack.
更一般地，脚本可以从输入流量中获取任何所需的属性和统计数据。事实上，Zeek 的所有默认输出都来自发行版中包含的脚本。 Zeek的语言具有广泛的特定领域类型和支持功能。至关重要的是，Zeek的语言允许脚本随着时间的推移保持状态，使它们能够跟踪和关联跨连接和主机边界观察到的内容的演变。 Zeek脚本可以生成实时警报，还可以根据需要执行任意外部程序。人们可以使用此功能来触发对攻击的主动响应。