Uncategorized

GUIMark Benchmark and Rendering Engine theory

Home | Detailed Analysis | Benchmark and Rendering Engine theory

Breaking down the test

Before creating the reference design I came up with the following rules to influence the overall benchmark. I wanted to make sure that whatever I created, the results would create meaningful data for both end designers and developers as well as platform engineers at Adobe or Sun or Mozilla to be able to use.

  1. The max draw rate for an application is fixed at 60 frames per second. Since mainstream LCDs are fixed at 60 hertz, most applications have no need to render above 60 frames per second and in general, the OS will throw away any draw requests it receives above that rate. While there are a select number of people still using CRTs and TV manufacturers are pushing 120hz technology, it’s a waste of cpu to generally create animations above 50 to 60 fps. With this in mind its important that no benchmark comes close to this cap. Its very difficult to visually distinguish between an animation running at 50 fps and one running at 60 fps. So the closer I can force the framerate down to 10 fps, the more visual difference there will be between the technologies. Unfortunately I underestimated the optimization capabilities of the Flash engineers.
  2. Animations should run on fixed time as opposed to relative time. One of the important distinctions in the GUIMark test case is that at high framerates, the test runs smoother, not faster. The heartbeat-style animation of the test will always take 2 seconds to perform, whether running at 5 fps or at 50 fps. This gives users a good idea of how smooth they can expect their own transitions to animate.
  3. The test case should be hard to optimize. I took a WYSIWYG approach to the benchmark so that there are no clear optimization paths. Each technology is limited in some way by the effects that are required in the reference design.
  4. Code execution should have a negligible impact. The user code in each of the tests has almost zero impact on the test case. This is very evident when observing Silverlight running on javascript vs C#. My prediction was that I’d see no difference between the 2 code paths and it turned out to be right. I wouldn’t expect to see any improvements when porting the Flex project into a pure AS3 project either. This jsut means that the rendering engine is taking up 99% of the cpu time and we get a pretty good comparison between the different environments.
  5. The rendering pipeline should remain saturated. If the fps is below 60 and the CPU isn’t sitting at 100% while running the test then there’s either something wrong with the test or theres something wrong with the runtime.

Understanding 2D Rendering Subsystems

The rendering subsystem is typically very complicated and includes alot of interworking logic to interface with user code, internal display objects and OS apis to optimize the final output. You can generally expect the 3 following pieces to be present in one way or another inside each runtime.

  • Layout engine: The layout engine is responsible for calculating the location for objects to be displayed. This includes performing matrix transforms like scaling, rotations, and object positioning, as well as positioning text within a flowing text block: It is also responsible for calculating the entire parent child display object chain.
  • Drawing engine: The drawing engine creates internal bitmaps of each discreet display objects. It applies colors to gradients, determines antialiasing rules, and draws fonts.
  • Composition engine: The compositing engine takes the results of the layout engine and drawing engine and sends a final render to the OS to draw. The compositing engine determines the z-index order of objects, calculates final pixel values for alpha overlays and applies clipping rules.

Optimizations within the render subsystem usually come from the interactions between the different engines. For instance, the drawing engine has no need to draw outside a clipped area or beneath other z-index objects, and the compositing engine doesn’t need to update areas of the display not changed by the layout engine or draw engine.