Unfortunately, we do not have the luxury of delivering low resolution graphics in our GUIs like some mobile devices either. HD video requires HD graphics. The problem really comes from the traditional approach of using images:
1. Read JPG/PNG image from source (carousel, file etc)
2. Decode to a buffer in memory
3. Copy the decoded buffer to screen memory
In step 2 we must hold a buffer that is RGB(A)*x*y in size for each image. The traditional solution was therefore to have a small active pool of images, which causes the delayed drawing effect seen in some interfaces or to use smaller images. Higher compression in the original image does not help of course as the JPG image is uncompressed before use.
We must solve the problem at step 2. Essentially - how do we hold an image in memory that is compressed yet can be directly copied to video memory.
With Opengl ES the newer graphics pipeline for images looks like this:
1. Read JPG/PNG image from source (carousel, file etc)
2. Decode to a buffer in memory
3. Create an OpenGL formatted texture image in memory
4. Use the texture to create images in video memory through OpenGL
Apparently, we have exacerbated the problem. However, using a combination of common sense and a smart feature of OpenGL-ES we can reduce the necessary storage space by a factor of 6. OpenGL-ES supports several texture compression modes. This means that when creating an OpenGL texture,an internal storage format can be specified that is a compressed image - just what we need. The trick thus is:
1. Read JPG/PNG image from source (carousel, file etc)
2. Decode to a buffer in memory the image
3. Free the memory from the JPG image
4. Create an OpenGL formatted compressed texture image in memory
5. Free the image buffer
6. Use the compressed texture to create images in video memory through OpenGL
The idea is to keep only one copy of the image in a compressed format. The compression is about 6:1. Compression may be limited to 2D texture with no border - ideal for TV space picture carousels. OpenGL-ES 2.0 supports the compression formats.
One more trick will save us another 30% of memory. Images in OpenGL (compressed ones) support linear interpolation. This means that images can be enlarged up to about 1.5x without noticeable degredation in quality. Therefore if we know how large an image will be on screen we can create an optimal sized image which is around 70% of the size of the image to be displayed.
In summary then what have we saved:
1. Only one uncompression buffer is required and this is temporary.
2. Images can be 30% smaller in resolution thanks to opengl interpolation.
3. Images can be 6:1 compressed.
This means where we once used 10Gig for images, we now use 1.16 Gig. Nearly an order of magnitude saving.
