Adobe(R) FLASH(R) for the Digital Home

Finally after months of trying, the FLASH story on set top boxes is clearing up. For the near future Adobe will be licensing Adobe Flash Lite 3.1 to silicon chip vendors. It will run on high end offerings only. Adobe offers two licensing models: free and paid. The Free license must leave the FLASH engine open (upgradeable) whereas the paid license may enclose the engine in the stack at manufacturing time. This is an interesting conflict. Most middleware and hardware vendors (who write drivers) would not want a third party upgrading the platform I suspect. The testing and support costs could be large. There is no such thing as WHQL for set top boxes so the latest FLASH engine can be a wild card. Secondly its not totally clear what Open means. Perhaps it will mean more than upgrading in the future (delivering adverts on FLASH startup?). This is a new model for set top box vendors and cable operators. However Adobe are most keen for the free model to be taken up and may make the paid license high in order to discourage its uptake.

Which version of FLASH? It will be FLASH lite 3.1 which is a subset of FLASH 8 but with some video support added from FLASH 9/10. The most interesting added video functionality is H.264. The missing functionality from FLASH 8 is:

• Filters (blur, drop shadow, and so forth)
• Blend modes (add, subtract, multiply, and so forth)
• Enhanced strokes (miter, square, and so forth)
• Text as Link
• setTimeout
• _target
• Encoding per pixel alpha with video created with Flash 8 Professional (On2 VP6)
• Bitmap caching
• ActionScript objash Rects or methods
• Flash remoting

The roadmap is currently under NDA but will be available publically on October 4th so watch this space...

IBC 2009

Well, for once I'm excited by a trip to IBC. I'm looking forward to the 3D graphics innovations that should be there this year. If you read this blog and you are at IBC, pop over to my employers stand and say hello and we can discuss graphics until, as they say, the cows come home.
Maybe you think your company should be on this blog?
Maybe you think you shouldn't have been?

Alticast, stand 1.C35,BM2
Ask for Chris Thornborrow

See you there!

Intel Canmore

The Intel Canmore (CE 3100) is a system on a chip for DTV. It includes an 800Mhz processor, security hardware, dual 300Mhz DSP decoders for video, PCI Express, ATA, USB and Wifi. Phew! There is an in-depth article over at EETimes.

The important thing for this blog is the chip also includes a 3d core - the PowerVR from Imagination Technologies. It is OpenGL ES 2.0 and OpenVG compatible. Here is a video of Yahoo widgets running on a Canmore device.



The widgets are not entirely smooth, but perhaps this is an early demo. In the meantime, a real set top box was launched by Gigabyte back in June 2009 using the Canmore.



Here we see a video the set top box running a game using two controllers!



And this month sees the launch of Metrologicals Mediaconnect TV, also powered by Canmore. I hope to see this at IBC and do a more in-depth report on the device.

So Canmore is gaining some market traction in DTV at last but its fair to say these are high end devices and Canmore remains a high end device.

NXP announce chips using PowerVR for set top boxes

In a press release dated September 04 2009, NXP Semiconductors announce their latest range of processors for set top boxes, the catchily named NXP PNX847x/8x/9x. The interesting part for this blog is that the chips integrate the PowerVR architecture from Imagination Technologies.

The key may lie in the power management goal of NXP who say in the press release that the architecture is specifically designed to maximize energy efficiency. This may explain the choice of PowerVR which uses a region based architecture that helps to reduce power useage and has been very successful in the mobile sector.

Job: User Interface development for set top box

Synacor is advertising a number of roles, one of which is for a set top box developer with web development skills. You need to be based in America to apply.

SVG on set top boxes

SVG (Scalable Vector Language) is a family of specifications of an XML-based file formats for describing two-dimensional vector graphics with animation. It includes full and light versions (Tiny SVG) more suited for the mobile environment. Recent versions of the standard include images and video as well as scripting.

SVG is an open standard, human readable and appears to play head to head with FLASH. There are many comparisons of the technologies available on the web, written fom different points of view such as this one. However, the comparison states FLASH does not support filters which FLASH 10 now does.

The famous tiger image above is a GIF image. However, it is rendered from a vector language called SVG. My blog provider does not support SVG files or I could have embedded that image as an SVG file directly into the web page.

SVG supports Java and JSRs are available that give language bindings (eg JSR 287 language bindings for Tiny 1.2) that allow for creating SVG files dynamically and rendering them.

One proponent of SVG in the set top box world is IPTV solution provider Dreampark. Dreampark, based in Sweden, offer a scalable user interface for IPTV. I was able to see the demo at IPTV world Forum 2009. The demo uses non-rotated (axis aligned if you like) boxes and images and text. Using scaling some psuedo-3D was present. It was not clear if the text scaled from the demo. So it appears the SVG from Dreampark is connected directly to the blitter and limited to the functionality it provides if a smooth, fast user experience is required. Of course over time, like FLASH, the vector functionality, such as lines and polygons, will be available as 3D hardware supporting Open-VG hits the market.

Ikivo also supply SVG for mobile devices and offer Enrich Mobile TV as a product. This is a user interface suite based on SVG for mobile TV. An example from their user interface is shown below.


Adobe meantime support SVG output from their Creative Tool suite. This means that whichever solution is chosen Adobe still win, though of course FLASH offers more opportunity to Adobe for additional license fees. And there is one of the real deciding factor between these two vector graphics languages: business. The licensing model for FLASH on set top boxes is not clear to me yet and I wonder if it is clear to anyone. Designers most likely will pick FLASH if they can but just how much will it add to the cost of an STB?

So in conclusion, FLASH and SVG offer similar functionality. FLASH is proprietary but has momentum in the press and with chip manufacturers. There are however a few companies using SVG for user interfaces for TV, Dreampark being the most significant. The license model is unclear for FLASH and once this is known the business choice between SVG and FLASH will become much more obvious.

Tile or Region Based Rendering

Another mini-tutorial in the series aimed at bringing DTV experts up to speed on graphics.

PowerVR, used in i-phones is a tile based rendering 3d pipeline. Broadcom, who acquired Stellar Semi-conductors back in the year 2000 invested in the PixelSquirt architecture and are still using a variant of it today. This is also a form of tile or region based rendering.

So what is it and why is it useful for set top boxes and mobile devices?

A normal graphics pipeline performs the following:

for each triangle

• transform it and light it
  
• project to 2d
• clip it to the window

for each pixel in the triangle

• check visibility (usually done with z-buffer hardware) and if its visible
  
• interpolate lighting
• fetch textures to draw on it
• write the pixel with alpha blending if necessary

A region based architecture adds an extra stage by dividing the screen into regions which may be tiles of pixels or may even be scan lines or a number of scan lines.

The extra stage then becomes:
for each triangle

• transform it and light it
  
• project to 2d
• clip it to the window
• determine which regions the triangle enters and cache a reference
  

Now for each region each pixel of each triangle is processed as before. There are variations and optimisations on this scheme but this is the basic idea.

This seems to add more complexity. Obviously, cache memory for the triangles is needed or the triangles must be sent many time to the pipeline and each time clipped to the regions. This seems wasteful.

However, tile based rendering has major benefits:

1. A z-buffer need only be the size of a tile, vastly cutting down the required on chip memory and saving cost at the IP level.
2. Tile based rendering increases cache coherence for texture fetches and , of course, z-buffer references and this cuts bandwidth to main memory
3. Power use is cut down due to simpler 1 and 2 above
4. Performance can increase over traditional architectures because of cache coherence.

A recent study showed that tile based architectures can cut memory bandwidth by nearly 2x over traditional architectures.

In set tops, 3D IP is costly as it is not a base function of the device. Even 5$ added to the BOM of an STB is too much. Smaller IP is better IP. Power consumption is not as much of an issue as in mobile devices however, and so some of the advantages of tile based chips such as PowerVR are not relevant. This leaves us with the advantage that greater performance can be achieved in the pixel drawing part of the pipeline. However here, the depth complexity (how many polygons overlap) in an average user interface must be quite low (an average of 2.0 say). Therefore the extra work a tile based renderer must do to bin the polygons, or send them many times, may outweigh the benefits.

One last point. Tiles need a cache. When screens are small the number of caches is small but when screen reach full HD resolution there are a large number of caches for the triangles. A tile needs to be around 32x32 pixels in size so for full HD we need 240 caches - a considerable number.