VR requires the support of many components in modern phones. This starts with the sensor for recording the motion of the head, the CPU driving the VR application (and everything else in the background), the GPU doing the work for the VR application and the calculations for creating the VR corrected image, to the display showing the transformed content to you, the observer.
在现代手机中，虚拟现实需要许多组件的支持。首先是用于记录头部运动的传感器，驱动 VR 应用程序的 CPU (以及背景中的其他一切) ，为 VR 应用程序工作的 GPU，以及创建 VR 校正图像的计算，最后是显示转换后的内容给你这个观察者的显示器。
All those components need to work closely together to create that immersive experience everybody is talking about. In a lot of publicly available content, the time to achieve this is called motion to photon latency. Although this is a very generic term, it describes the problem very well: when will the change in view before my eyes eventually be recognized by my eyes and processed by my brain because of a head motion?
所有这些组件需要紧密合作，以创建每个人都在谈论的身临其境的体验。在许多公开的内容中，实现这一点的时间称为运动到光子的延迟。虽然这是一个非常通用的术语，但它很好地描述了这个问题: 什么时候我眼前的变化最终会被我的眼睛识别，并由于头部运动而被我的大脑处理？
In the real world, this happens instantly – otherwise you would run against walls all the time. However, creating this very same effect with a phone strapped to a head-mounted display turns out to be quite hard. There are different reasons for this: computers running at fixed clocks, the pipelining which serializes the processing of the data, and obviously the time it takes to render the final images.
在现实世界中，这种情况会立即发生——否则你就会一直撞墙。然而，把手机绑在头戴式显示器上，要创造出同样的效果是相当困难的。有不同的原因: 计算机运行在固定的时钟，流水线序列化的数据处理，显然需要时间来呈现最终的图像。
We as a GPU IP company are of course interested in optimizing our graphics pipeline to reduce the time it takes for the PowerVR graphics processor to render the content until it appears on the display.
作为一家图形处理器 IP 公司，我们当然有兴趣优化我们的绘图管线，以减少 PowerVR 图形处理器在显示内容之前呈现内容所需的时间。
Composition requirements on Android
Android 的组合要求
The final screen output on Android is rendered using composition. Usually there are several producers that create images on their own such as SystemUI which is responsible for the status bar and the navigation bar; and foreground applications that render their content into their own buffers.
Android 上的最终屏幕输出是使用合成呈现的。通常有几个生产者自己创建图像，比如负责状态栏和导航栏的 SystemUI，以及将内容呈现到自己的缓冲区中的前台应用程序。
Those buffers are taken by a consumer that arranges them and this arrangement is then eventually displayed on the screen. Most of the time this consumer is a system application called SurfaceFlinger.
这些缓冲区由使用者进行排列，然后这种排列最终显示在屏幕上。大多数情况下，这个消费者是一个名为 SurfaceFlinger 的系统应用程序。
SurfaceFlinger may decide to put those buffers (in the correct arrangement) directly on the screen if the display hardware supports that. This mode is called Hardware Composition and needs direct support by the display hardware. If the display doesn’t support a certain arrangement, SurfaceFlinger will use a framebuffer the size of the screen and render all images with the correct state into that framebuffer by using the GPU. That final composition render will then be shown by the display as usual.
如果显示硬件支持，SurfaceFlinger 可能会决定将这些缓冲区(以正确的方式)直接放在屏幕上。这种模式称为硬件组合，需要显示硬件的直接支持。如果显示器不支持某种安排，SurfaceFlinger 将使用屏幕大小的帧缓冲区，并使用 GPU 将所有具有正确状态的图像渲染到该帧缓冲区。最终的合成渲染将像往常一样显示。
Both the producers and the consumer are highly independent – actually they are even different processes which talk to each other using inter-process communication. Now, if one is not careful, all those different entities will trample over each other all the time.
生产者和消费者都是高度独立的——实际上他们甚至是不同的过程，用行程间通讯互相交流。现在，如果一个人不小心，所有这些不同的实体将践踏对方所有的时间。
For example a producer may render into a buffer which is currently displayed to the user. If this happens the user of the phone would see this as a visual corruption called tearing. One part of the screen still shows the old content and the other part already shows the new content. Tearing is easily identifiable by seeing cut-lines.
例如，生产者可以呈现到当前显示给用户的缓冲区中。如果发生这种情况，手机用户会将其视为一种称为撕裂的视觉损坏。屏幕的一部分仍然显示旧内容，另一部分已经显示新内容。撕裂很容易通过看到切割线来识别。
To prevent tearing, two key elements are necessary. The first is proper synchronization between all parties and the second is double buffering.
为了防止撕裂，两个关键因素是必要的。第一个是所有各方之间的正确同步，第二个是双缓冲。
Synchronization is achieved by using a framework called Android Native Syncs. Android Native Syncs have certain criteria which are important in the context of how they are being used on Android. They are system global syncs implemented in kernel space which are easily shareable between different processes by using file descriptors in usermode.
同步是通过使用一个名为 Android NativeSyncs 的框架来实现的。Android 本地同步有一定的标准，这些标准对于 Android 上如何使用它们非常重要。它们是在内核空间中实现的系统全局同步，通过在用户模式中使用文件描述符，可以方便地在不同进程之间共享。
They are also non-reusable binary syncs that allow just the states of “not signaled” and “signaled” and have only one state transition of going from “not signaled” to “signaled” (and not the other way around.)
它们也是不可重用的二进制同步，只允许“无信号”和“有信号”的状态，并且只有一个从“无信号”到“有信号”的状态转换(而不是反过来)
Double buffering allows the producer to render new content while the old content is still being used by the consumer. When the producer has finished rendering, the two buffers are switched and the consumer can present the new content while the producer starts rendering into the (now old) buffer again.
双缓冲允许生产者在使用者仍在使用旧内容时呈现新内容。当生产者完成渲染后，两个缓冲区被切换，使用者可以显示新的内容，而生产者开始再次渲染到(现在已经旧的)缓冲区中。
Both mechanisms are necessary for the fluid output of the user interface on Android, but unfortunately they also have a cost which is additional latency.
这两种机制对于 Android 上用户界面的流动输出都是必要的，但不幸的是，它们也有额外的延迟成本。
Having double buffering means the content rendered right now will be visible to the user one frame later. Also the synchronization prevents access to anything which is on screen right now. To remove this particular latency the idea of single buffering can be used. Basically it means render always to the buffer which is on screen. Obviously to make this happen the synchronization also needs to be turned off.
具有双缓冲意味着现在呈现的内容将在一帧之后对用户可见。此外，同步阻止访问任何在屏幕上现在。为了消除这种特殊的延迟，可以使用单缓冲的思想。基本上，它意味着渲染总是在屏幕上的缓冲区。显然，要实现这一点，还需要关闭同步。
We implemented this feature in the KHR_mutable_render_buffer EGL extension ratified by the Khronos Group just this March.
我们在 Khronos Group 今年 3 月批准的 KHR _ mutable _ render _ buffer EGL 扩展中实现了这个特性。
This extension needs support from both the GPU driver and the Android operation system. As I explained earlier, Android goes a long way to prevent this mode of operation because it results in tearing.
这个扩展需要 GPU 驱动程序和 Android 操作系统的支持。正如我前面所解释的，Android 在防止这种操作模式方面做了很多工作，因为它会导致撕裂。
So how can we prevent tearing when running in the new single buffer mode?
那么，在新的单缓冲区模式下运行时，如何防止撕裂呢？
Screen technologies
屏幕技术
To answer this question we need to look into how displays work. The GPU driver posts a buffer to the display driver which is in the format the display can understand. There may be different requirements for the buffer memory format, like special striding or tiling alignments.
为了回答这个问题，我们需要研究显示器是如何工作的。GPU 驱动程序向显示驱动程序发送一个缓冲区，该缓冲区采用显示器可以理解的格式。对于缓冲区内存格式可能有不同的要求，比如特殊的跨距对齐或平铺对齐。
The display normally shows this new buffer on the next vsync. Assuming the display scans out this memory from top to bottom the vsync or vertical sync is the point in time when the “beam” goes from the bottom back to the top of the screen.
显示器通常会在下一个 vsync 上显示这个新的缓冲区。假设显示器从上到下扫描这个内存，那么 vsync 或者垂直同步就是“光束”从底部返回到屏幕顶部的时间点。
Obviously nowadays there is no beam anymore, but this nomenclature is still in use. Because at this point in time the display does not scan out from any buffer, it is safe to switch to a new one without introducing any tearing. The display does this procedure in a fixed time interval defined by the screen period.
很明显，现在已经没有光束了，但是这个术语仍然在使用。因为此时显示器不会从任何缓冲区扫描出来，所以切换到新的缓冲区是安全的，不会引起任何撕裂。显示器以屏幕周期定义的固定时间间隔执行此过程。
A common screen period is 16.7ms. This means the image on screen is updated 60 times per second.
一个常见的屏幕周期是 16.7 毫秒，这意味着屏幕上的图像每秒更新 60 次。
Modern phones usually have an aspect ratio of 16/9 and their screen is mounted so that the scan out direction is optimal for portrait mode, because this is the mode most users will use their phone on a daily basis.
现代手机的长宽比通常为 16/9，而且它们的屏幕是安装在屏幕上的，因此扫描出来的方向对于纵向模式来说是最佳的，因为这是大多数用户每天使用手机的模式。
For VR this changes and the VR application will run in landscape resulting in a scan out direction from left to right.
对于虚拟现实，这个变化和虚拟现实应用程序将运行在横向导致从左到右的扫描方向。
This is important when we look now at how to update the buffer while it is still on the screen.
当我们现在考虑如何在缓冲区仍然在屏幕上时更新它时，这一点非常重要。
Strip rendering
条带渲染
When the display is scanning out the buffer, it is doing so at a constant rate. The idea of strip rendering is to change that part of the buffer which is currently not scanned out. There are two different strategies available. We could try to change the part of the screen where the scan out happens next.
当显示器扫描出缓冲区时，它是以恒定的速率这样做的。条带渲染的思想是更改当前未扫描出的缓冲区部分。有两种不同的策略。我们可以试着改变屏幕上接下来扫描出来的部分。
This is called beam racing, because we try to run in front of the beam and update the part of the memory which is just about to be shown next. The other strategy is called beam chasing and means updating the part behind the beam.
这就是所谓的光束竞赛，因为我们试图跑在光束前面，并更新记忆的一部分，即将显示下一步。另一种策略称为光束追踪，意味着更新光束背后的部分。
Beam racing is again better for latency, but at the same time harder to implement. The GPU needs to guarantee to have finished rendering in a very tight time window. This guarantee can be hard to fulfil in a multi-process operating system where things happen in the background and the GPU also may render into other buffers at the same time. So the easier method to implement is beam chasing.
束流竞赛再次更好的延迟，但同时更难以实现。GPU 需要保证在一个非常紧的时间窗口内完成渲染。在一个多进程操作系统中，这种保证可能很难实现，因为事情发生在后台，并且 GPU 也可能同时呈现到其他缓冲区中。因此，最容易实现的方法是波束追踪。
The VR application asks the display for the last vsync time and adjusts itself to it to make sure the render has the full screen period time for the presentation. To calculate the time for a strip to start to render, we need to define the strip size first. The strip size and therefore the number of strips to use is implementation defined. It depends on how fast the display scans out and how fast we are able to render each strip. In most cases two strips are optimal. In VR this also has the benefit of having one strip per eye (remember we scan out from the left to the right) which makes the implementation much easier. In the following example we still use four strips to see how this works in general. As I mentioned earlier, we have 16.7ms to render the full image.
VR 应用程序要求显示最后的 vsync 时间，并自我调整以确保渲染具有演示文稿的全屏时间。为了计算条带开始渲染的时间，我们首先需要定义条带的大小。条带的大小和使用条带的数量是实现定义的。这取决于显示器扫描出来的速度以及我们渲染每个条带的速度。在大多数情况下，两条带是最佳的。在虚拟现实中，这也有一个好处，每只眼睛只有一个条带(记住我们从左向右扫描) ，这使得实现更加容易。在下面的示例中，我们仍然使用四个带来了解一般情况下是如何工作的。正如我前面提到的，我们有 16.7 毫秒渲染完整的图像。
Consequently we have 4.17ms to render each strip. The VR application waits 4.17ms after the last vsync and then starts rendering the first strip. Now it waits again 4.17ms (which is 8.34ms after the last vsync) before rendering the second strip. This gets repeated until all four strips are rendered and the sequence can start over again. Obviously we have to take into account how long each render submission takes and adjust our timing accordingly. Using absolute times has been shown to be the most accurate method to use.
因此，我们有 4.17 毫秒渲染每条。VR 应用程序在最后一次 vsync 之后等待 4.17 ms，然后开始呈现第一条带。现在，它再次等待 4.17 ms (比上次 vsync 晚了 8.34 ms) ，然后才显示第二条带。这个过程会一直重复，直到所有四个条带都被渲染，并且序列可以重新开始。显然，我们必须考虑到每个渲染提交需要多长时间，并相应地调整我们的时间。使用绝对时间已被证明是最准确的方法使用。
Final thoughts
最后的想法
Reducing latency in VR applications on mobile devices is one of the major challenges facing developers today. This is different from desktop VR solutions which benefit from higher processing power and usually don’t encounter any thermal budget limitations.
减少移动设备上虚拟现实应用程序的延迟是当今开发人员面临的主要挑战之一。这不同于桌面 VR 解决方案，后者受益于更高的处理能力，并且通常不会遇到任何热预算限制。
Make sure you also follow us on Twitter (@ImaginationTech) for more news and announcements.
请确保您也在 Twitter (@ImaginationTech)上关注我们，以获得更多的新闻和公告。
标签：render,缓冲,screen,VR,Strip,缓冲区,Android,display From： https://www.cnblogs.com/mikaelzero/p/16929718.html