alundeb wrote:
I am not Matt, but as a programmer, that doesn\'t make sense me.
An array of 4096 indexed 14 bits values, where only half of them are used, does not make it a 13 bits limitation. Antisthenes confuses the index and the content. It may still be a fully valid map of 2048 true 14 bits values.
He actually said:
The 4096-entry curve[] array is indexed with the 11-bit values.
So the 11-bit values that come out of the delta compression are used to index this LUT, which contains 13-bit values. Only half of the LUT is populated/used. It\'s declared with a size of 4096 as it\'s used for different purposes in other parts of the program.
It also happens to contain 13-bit values. Once the LUT is applied the result is bitshifted right to get a 14-bit value (with a zero LSB). The LUT could just have easily been populated with 14-bit values (this would in fact have been marginally more efficient). How ever it would have then started 0,2,4,6,8,10 instead of 0,1,2,3,4,5 - there would be no difference in practice.
alundeb wrote:
The mimimum value step is determined by the gamma curve, or WHAT 14 bits values you initilaize with.
Yes that is the crux of the issue. I think everyone is arguing the same thing but being bogged down on details.
alundeb wrote:
I am not Matt, but as a programmer, that doesn\'t make sense me.
An array of 4096 indexed 14 bits values, where only half of them are used, does not make it a 13 bits limitation. Antisthenes confuses the index and the content. It may still be a fully valid map of 2048 true 14 bits values.
He actually said:
The 4096-entry curve[] array is indexed with the 11-bit values.
So the 11-bit values that come out of the delta compression are used to index this LUT, which contains 13-bit values. Only half of the LUT is populated/used. It\'s declared with a size of 4096 as it\'s used for different purposes in other parts of the program.
It also happens to contain 13-bit values. Once the LUT is applied the result is bitshifted right to get a 14-bit value (with a zero LSB). The LUT could just have easily been populated with 14-bit values (this would in fact have been marginally more efficient). How ever it would have then started 0,2,4,6,8,10 instead of 0,1,2,3,4,5 - there would be no difference in practice.
Sep 17, 2015 at 07:42 AM
Previous versions of Matt Grum's message #13202757 « Uncompressed 14 bit for A7sii and A7rii »
