Monday, August 3, 2009

I want FLASH, why do you talk about OpenGL?

There is some confusion over FLASH and its relation to OpenGL-ES hardware for set top boxes. This mini-tutorial attempts to clarify and in so doing, explain why FLASH needs OpenGL-ES hardware and why FLASH on set top boxes without OpenGL-ES hardware runs slowly.

FLASH is a vector language which, at some basic level, draws lines, polygons, shaded areas, curves, text and of course images. FLASH also includes a lot of blending modes (more on that in a later article) which dictate how what is being drawn is combined with what is already drawn.

Graphics hardware for set tops today is mainly blitter based. The blitter can draw images and solid filled areas and has very limited blend modes but cannot draw arbitrary polygons, lines, curves etc. Thus today only part of FLASH can be accelerated and more often than not, FLASH must drop back to software to render, except in very limited special cases. Certainly today "wild" FLASH, FLASH from the internet, not under the control of the operator, will drop back to software to run.

Simply put, FLASH needs graphics hardware to run well, and, needs more than just a blitter.

OpenGL 3D hardware is pervasive. It has been developed over about 30 years and is now cost effective and highly optimised. OpenGL itself is a flexible and powerful set of drawing commands.

Thus layering FLASH on OpenGL hardware, rather than having dedicated FLASH hardware makes sense.

One complication is the version of FLASH and how the layering is achieved. Some versions of FLASH are 2D (eg FLASH lite). Layering 2D in 3D OpenGL hardware is laborious. Fortunately the folks who design OpenGL (Khronos) have designed a 2D vector language which maps nicely to OpenGL-ES called OpenVG - Open Vector Graphics. Thus chip manufacturers are providing a double whammy of OpenGL-ES and OpenVG. Thus porting FLASH to OpenVG is more realistic for early/lighter versions of FLASH. The diagram below shows this.



Later versions of FLASH have 3D effects built in. Thus it makes more sense to layer directly on OpenGL-ES. Be aware though that just because a company has a FLASH 7 player, it does not mean they can easily make the jump to FLASH 10 which requires sound, per pixel image processing and 3D, all of which are large jumps in functionality. It is very likely they will have to re-port from OpenVG to OpenGL-ES.

Sound will logically be best supported by chips with floating point co-processing such as those from STMicro. However, fixed point can be used. Per pixel image processing requires shaders (to be explained in a later article). Shaders come with OpenGL-ES 2.0 and are not present in OpenGL-ES 1.1 hardware. Even so, do not assume that ES 2.0 hardware will run per pixel effects WELL. Demand a demo. This is because the cores on the chips may be underpowered when it comes to shaders for a generation or even two. This was the case in PCs and with the power contraints placed on STB manufacturers, may be even more true in set top boxes...we will have to wait and see.

In conclusion, on most of set top boxes today, FLASH will run slowly in software unless the FLASH is authored very carefully. This is because a blitter simply cannot render most of what is required. Once OpenGL hardware is everywhere, FLASH graphics will run much faster though FLASH 10 will have some slow areas such as sound and per pixel image effects.

Then its a question of how fast the logic engine inside FLASH runs of course...but thats not for a graphics blog...

No comments:

Post a Comment