char cc = {52, -66, -55, -98, 26, -10, 45, 5, -34, 58, -63, 15, -57, 60, 43, 21};

I was wondering if you can reduce this 16 bytes.

And also we will be interested in seeing the unsort source you mentioned above and we test it at our research center.

As stated before, there’s data that just cannot be compressed. In my opinion, though, MOST of data we believe it’s random it’s not. It can be further compressed. The only problem is that we are, currently, unable to devise models adequately to compress them.

simple example using factorial(2):

data = ’10′; // length = 2.

encoding:
newdata = data[0]; // length = 1.

decoding:
data = newdata;
if data = ’0′ then data = data + ’1′ else data = data + ’0′;

Encoding and decoding factorials of (256) (or even factorial(2^31)) can be done quite easily using a 32-bit arithmetic encoder (like Mark Nelson’s for example) and a MoveToBack model lowering the high bound at each step.

Thanks for your comment, I will be posting my next part of the subject this weekend.
We can discuss after this post.

Just to you inform you that
1. If all 256 are unique(in each 256block) I can compress using reverse merge sort with approximately 10% to 50%.
(Depending on the order of items, if it is in ascending order we can achieve 50% compression, worst case if it in descending order.)
2. If the unique numbers(in each 256block) are between 230 to 255 then we can compress around 1% to 10%.
3. But if the uniqueness is between 128 to 230, I couldn’t get any solution so far.

In fact Million Random digit contains around 150 unique values and remaining duplicates.

Thanks & regards
Keshav K Shetty

After 2 year research in this field (90% looks like your work description), I consider that only useful thing about random data value uniqueness is that – there are limits of compressible and un-compressible data.
I did not find information regarding such uniqueness compressor, but I found/calculate some numbers.
Let’s take 8b sample (all 256 values from 0-255 in random order):
1) the best (cost less bits) method to store such unique values is log(256!, 2) (base 2).
2) the margins I found for compressible data are:
a) about less than 130 unique values in 256 value sequence (cmpressible using arithmetic coding and similar);
b) 256 unique values or more than about 180 unique values in 256 value sequence (regarding our research).

I did not write any paper about that but if you will go further you will face those values. What about your research so far, did you found some more facts?

P.S. I’m not mathematician and English is not my native

God gave us our world and indeed, he loves us, even sending Jesus to help us get back on track. So don’t ignore him! For he really is God come to earth. Instead honor him as God for that is exactly who he is.

Now about compression.

Won’t you be surprised because what you believe to be impossible, well, that’s exactly what I’ve done.

I like Mark but if he actually said what you claim (I haven’t checked,) well then, he was wrong. In point of fact, while the basic points you (or he,) makes are quite true, they shouldn’t be extended to imply that repeated compression of random input is impossible.

One more time. No, it’s not impossible, I do it.