In my last post, I was talking about image compression methods, mainly DCT transform based along with colour transform based. I’ve now combined these together and have managed to break the 6fps barrier for video over UART! I’ve implemented 2:1:1, 4:1:1 and 8:1:1 subsampling (I think I can call it that?) with reasonable performance! The colour is poor as every but the frame rate is good. The DCT is executed on YCgCo pixels as opposed to RGB pixels. These are recovered on the C# program side and displayed.
To implement this subsampling, for every DCT block from an 8×8 YCgCo pixel block, I write every Y channel. For 2:1:1 sampling, I write the Cg and Co channels for every other Y channel. In memory, this would look like Y1Cg1Co1 Y2 Y3Cg3Co3 Y4 Y5Cg5Co5 etc. This drastically reduces transfer requirements and space consumption. I also further reduce CgCo quality by only storing the top 2×2 DCT values for these channels. This compresses a block from 3*3*3 =27 bytes to 3*3 + 2*2*2 = 17 bytes (still padded to 18 bytes to ensure a 16bit word is written to memory). A block row of no sub sampling would equal (640/8)*(3*3+2*2*2) 1360 bytes. With there being 480/8 = 60 rows of these 8×8 blocks, this is equivalent to an image size of 81.6kB. Pretty small but not as small as I’d like!
With 2:1:1 sub sampling (like the demonstration above), the 80 blocks in a row would alternate in size between 17 bytes and 3*3 9 bytes (rounded up to 18 bytes and 10 bytes). Using 2:1:1 sub sampling allows for block row sizes of 1120 bytes, giving a total image size of 67.2kB. Increasing subsampling to 4:1:1 gives an image size of 57.6kB and finally, increasing to 8:1:1 (pretty bad!) gives an image size of 52.8kB. Every nugget of compression adds up to faster frame rates through UART with a reduction in quality for the level of subsampling used.
2:1:1, 4:1:1 and 8:1:1 subsampling, note the development of artifacts for higher subsamplings!
This is only a short post to add onto my earlier post.