As I mentioned earlier – it is hard or impossible(as of now) to compress million random digit. As per my analysis million random digit file contains around 90 – 110 bytes duplicates or repeated number within every block of 256 bytes. When input data is very pure or near pure(I mean uniqueness) or when input data is highly polluted or noise(I mean duplicates), then it is easy to compress. May be we can borrow the idea from chemistry. i.e if you refer periodic table lighter elements can be combined using nuclear fusion(Stars) or higher elements can be broken using nuclear fission(Uranium). In both cases energy released which we can relate to compression, however if you refer periodic table you will understand that elements with highly stable nucleus hard to break or fuse. e.g: Lead.

Same pattern I can see in million random digit where if uniqueness reaches extreme high or extreme low we can easily compress the data.

My so far developed compressor could achieve compression with doubles less than 42 or more than 156 in every 256 bytes.

Before I go further about my compression theory, I have to write one more article about one bit difference variable encoder. This is one of the known algorithm based on probability or possibilities.

About bijective you might have observed it is one on one mapping, however this fails to address relation within elements of donor domain set or acceptor domain set.

The one bit variant encoding based on the possibilities. i.e if we have 4 unique possibilities then we need 2 bits (00,01,10,11) to represent all possible values. How many bits do we require when we have 5 possibilities or say 6 possibilities?

In one of my earlier article I mentioned usage of insertion sort or bubble sort stating using this technique we can represent for every possible combination we will have 1792 bits or 224bytes. But I need to re factor this statement. We need much lesser bits.

Assume that you have a sorted list of 256 all unique values, when a random input comes lets say 26, you pick from the initialized list, actually you pick the position, Since it is first input and there are 256 possibilities we need eight bit to represent the position. For next random input we need to chose among remaining 255 possibilities, in this case exception for last position we need eight bit. Here it depends on the position which we pick from remaining elements. We can calculate required bits for best case scenario(when input are in sorted order) and worst case(when input are in reverse sorted order) as below.

If you refer the below illustration you will quickly understand that as possibilities changes required bits changes by maintaining uniqueness of bit stream.

The cells in yellow color are one bit diff encode based first entry in each row.

Using above calculation you can represent using 1545-1793bits(193.125-224.125bytes) for every 256 unique bytes.

Next article I will explain how I achieved compression for 42doubles.

Some of the readers asked how reverse merge sort (merge unsort) can be used to represent 256 unique values using 128bytes(best case) or 224.25(worst case).

As illustrated in previous article using merge sort we sort the random input and store bit information of the list from where we picked the smaller number. Actually we are reshuffling the original position and stored bit information represents how a input changed its position from original list to sorted list. Lets take an example of a random array of 16 numbers varying from 0…15.

As shown in above image after sorting, the position are reshuffled and stored bit information exactly represent from where the each element moved.

The encoding code is recursive function, to start applying merge from two list of size 2 to merge the two list of 128. You can refer the mergesort in action here.

The merge sort encoding code given below(C++).

   void Transform::mergeSort(unsigned int beginPosition, unsigned int endPosition) {
                unsigned int midPosition = beginPosition + (endPosition-beginPosition)/2;
                if (midPosition>beginPosition && midPosition<endPosition) { // Recursive call until list is of size 2
                        mergeSort(beginPosition,midPosition);
                        mergeSort(midPosition+1,endPosition);
                }
                unsigned int elementsInEachList = (endPosition-beginPosition+1)/2;
                unsigned char list1[256];
                unsigned char list2[256];
                for(unsigned int i=beginPosition;i<beginPosition+elementsInEachList;i++) {
                        list1[i] = transformedByte[i];
                }
                for(unsigned int i=midPosition+1;i<midPosition+1+elementsInEachList;i++) {
                        list2[i] = transformedByte[i];
                }
                unsigned int pendingInList1 = elementsInEachList;
                unsigned int list1Pointer = beginPosition;
                unsigned int pendingInList2 = elementsInEachList;
                unsigned int list2Pointer = midPosition+1;
                unsigned int mainArrayPointer = beginPosition;
                while(pendingInList1>0 && pendingInList2>0) {
                        if (list1[list1Pointer]<list2[list2Pointer]) {
                                transformedByte[mainArrayPointer] = list1[list1Pointer];
                                mainArrayPointer++;
                                list1Pointer++;
                                pendingInList1--;
                                bitWriter.write(false); // Store bit information 0 as item picked from first list
                        } else {
                                transformedByte[mainArrayPointer] = list2[list2Pointer];
                                mainArrayPointer++;
                                list2Pointer++;
                                pendingInList2--;
                                bitWriter.write(true); // Store bit information 1 as item picked from second list
                        }
           }
           if (pendingInList1>0) {
                        while(pendingInList1>0) {
                                transformedByte[mainArrayPointer] = list1[list1Pointer];
                                mainArrayPointer++;
                                list1Pointer++;
                                pendingInList1--;
                        }
                } else {
                        while(pendingInList2>0) {
                                transformedByte[mainArrayPointer] = list2[list2Pointer];
                                mainArrayPointer++;
                                list2Pointer++;
                                pendingInList2--;
                        }
           }
        }

As you observed above when items in one list gets over (either first list or second list) for remaining element we don’t require sorting bit information.

Please note in above code random input is stored in transformedByte variable & initilized in separate method for each 256byte input. The object bitWriter is a separate class of type BitWriter(Utility class) which is initialized in constructor. BitWriter will accumulates bit information and writes to output file when accumulated bits crosses 1 byte.

The merge sort decoding code given below(C++).

  void Transform::reverseMergeSort(unsigned int beginPosition, unsigned int endPosition) {
        unsigned int midPosition = beginPosition + (endPosition-beginPosition)/2;
        if (midPosition>beginPosition && midPosition<endPosition) { // Recursive call until list is of size 2
                mergeSort(beginPosition,midPosition);
                mergeSort(midPosition+1,endPosition);
        }
        unsigned int elementsInEachList = (endPosition-beginPosition+1)/2;
        unsigned char list1[256];
        unsigned char list2[256];
        for(unsigned int i=beginPosition;i<beginPosition+elementsInEachList;i++) {
                list1[i] = inputPositionArray[i];
        }
        for(unsigned int i=midPosition+1;i<midPosition+1+elementsInEachList;i++) {
                list2[i] = inputPositionArray[i];
        }
        unsigned int pendingInList1 = elementsInEachList;
        unsigned int list1Pointer = beginPosition;
        unsigned int pendingInList2 = elementsInEachList;
        unsigned int list2Pointer = midPosition+1;
        unsigned int mainArrayPointer = beginPosition;
        while(pendingInList1>0 && pendingInList2>0) {
                bool nextBit = bitReader.genNextBit(); // Read bit information from sorted encoding
                if (nextBit==false) {
                        inputPositionArray[mainArrayPointer] = list1[list1Pointer];
                        mainArrayPointer++;
                        list1Pointer++;
                        pendingInList1--;
                } else {
                        inputPositionArray[mainArrayPointer] = list2[list2Pointer];
                        mainArrayPointer++;
                        list2Pointer++;
                        pendingInList2--;
                }
        }
        if (pendingInList1>0) {
                while(pendingInList1>0) {
                        inputPositionArray[mainArrayPointer] = list1[list1Pointer];
                        mainArrayPointer++;
                        list1Pointer++;
                        pendingInList1--;
                }
        } else {
                while(pendingInList2>0) {
                        inputPositionArray[mainArrayPointer] = list2[list2Pointer];
                        mainArrayPointer++;
                        list2Pointer++;
                        pendingInList2--;
                }
        }
}

Before calling reverseMergeSort inputPositionArray is initialized with 0,1,2.. as default initial position. The object bitReader is a separate class of type BitReader(Utility class) which is initialized in constructor and provides API to read a bit at a time(not byte), which internally reads a byte and gives a bit at a time. These utility classes are developed by me exclusively for above purpose.

After exiting reverseMergeSort the inputPositionArray does not represent original input, instead it represent the positions of input in original data. After exiting we have to go thru one more loop to construct original input as below.

   for(int i=0; i<256;i++) {
       inputData[inputPositionArray[i]] = i;
   }

Now inputData contains original data.

I will inform in next article how I compressed random data in which we have maximum 42 duplicates(non unique) for every 256 input byte.

An oxymoron is usually defined as a phrase in which two words of contradictory meaning are brought together:

Read till end

1) Clearly misunderstood

2) Exact Estimate

3) Small Crowd

4) Act Naturally

5) Found Missing

6) Fully Empty

7) Pretty ugly

Seriously funny

9) Only choice

10) Original copies

And the Mother of all……
.
.
.
.
.
.

11) Happily Married

Visit Bangalore city traffic police

Check your vehicle traffic violation fines here “Search for Traffic Violations”

Understand and follow all traffic sign in Bangalore and India – Traffic sign

Various spot fine – List of Traffic offences, section of law and fine amount

Traffic Provisions – Traffic Do’s & Don’ts

Traffic Road Markings – Traffic Do’s & Don’ts

Traffic Rules and Regulations – Traffic Do’s & Don’ts

Traffic Lights – Traffic Do’s & Don’ts

Traffic Advice – Traffic Do’s & Don’ts

Tips on Good Driving – Traffic Do’s & Don’ts

You can download pdf version here

Introduction

The Rubik’s Cube (also spelled Rubick’s Cube, or Rubix Cube) is one of the most puzzling toys of all time. It ranks as one of the most cherished 80s icons and

few people have ever claimed to solve it on their own. In fact, most solutions have come from mathematicians using group theory. But it is solvable. Trial and

error will get you nowhere, so below is a 7 step process that will get you to sweet, sweet resolution.

Using this method, even an idiot can solve a cube in seven basic steps. Note the diagram below. Each face of the cube is assigned a letter. During the following

steps, specific faces require a sequence of twists (or quarter turns). The letter “i” means inverse, or counter-clockwise twists. For example, in the following

sequence

Ri U Fi Ui, you would rotate the Right face counterclockwise a quarter turn, the Upper face clockwise a quarter turn, the Front face

counterclockwise a quarter turn, and the Upper face counterclockwise a quarter turn. Before you start each move, make sure your thumbs are on the F side of

the cube to ensure consistent orientation for all the sequences. To turn a face in the right direction, imagine that you are facing that side of the cube. If you mess

up along the way, just restart from Step 1. Let’s begin.

STEP 1: Solve the Upper Green Cross

To solve the green cross, you have to solve the green edge pieces on their own. This should be easy to figure out on your own. Should you ever have an edge piece

in the correct place but flipped the wrong way, use this step to flip it without affecting the other three green edges: Hold the cube with the piece in the

upper-right position as in the diagram, and do the sequence Ri U Fi Ui. The edge piece should now be solved, and you can work the next edge piece until the

cube looks like the right diagram below.

STEP 2: Solve the Green Corners

Find the corner piece in the bottom layer that belongs on top. Turn the bottom layer until the piece is directly below its home in the top layer. Hold the cube with

the piece on the lower-front-right and its home at the upper-front-right and then do the sequence Ri Di R D 1,3, or 5 times until that corner is solved. If you find

a corner piece that’s already in the top layer but the wrong spot or flipped the wrong way, hold the cube with the piece in the upper-front-right position and do Ri

Di R D once. Now the piece is on the bottom and ready to be solved using the Ri Di R D sequence.

STEP 3: Solve the Middle Layer Edges

Flip the cube so green is on the bottom. Find the yellow-red edge piece. If it’s on top, turn it so it matches one of the diagrams below. Then do the corresponding

sequence to solve it. If the red-yellow edge is somewhere in the middle layer but it’s in the wrong place or flipped the wrong way, hold the cube so the edge is in

the front-right position and do either sequence once: U R Ui Ri Ui Fi U F or Ui Fi U F U R Ui Ri (This may require that you rotate the cube to a new face).

After the move, the piece is in the top layer and you can solve it as described above. Repeat for the other 3 middle-layer edges.

STEP 4: Solve the Upper Blue Cross

Turn the top layer until the edges match one of the diagrams. Repeat the following sequence as many times as it takes to get a blue cross: F R U Ri Ui Fi.

STEP 5: Solve to Top Edges

Hold the cube with red in front. Turn the top layer until the red and blue edge piece is solved as in the diagram, and then repeat the following sequence until the

yellow and blue edge piece is also solved on the right side: R U Ri U R U U Ri. Now turn the whole cube so that white is the front face. If the top white edge

isn’t solved, do the sequence again followed by and extra U.

STEP 6: Solve the Top Corners

Find a corner piece that’s in the right place and hold the cube with that piece above your right thumb. Don’t turn the top layer at all, as it well mess up all the

effort from Step 5. Do the following sequence once or twice to put the rest of the corners into place: U R Ui Li U Ri Ui L. If you can’t find a corner piece in the

right place, just do the sequence once before you start.

STEP 7: Solve the Top Corners (Again)

You’re almost done, you beautiful mind, you. Hold the cube with the red in front. Keep turning the top layer until the upper-front-right-corner needs to be flipped

to have blue on top like in the diagram. Do the sequence Ri Di R D 2 or 4 times to get blue into position. The cube will get scrambled in the process, but don’t

worry. With red still in front, keep turning the top layer and repeating the sequence to flip the upper-right-corner pieces.

STEP 8: Solve the Top Corners (Again – Special case)

Try this if you land up with side two corner piece in right direction, but wrong position. Check the ‘before’ image

Autorickshaw drivers in Bangalore are increasingly being associated with hooliganism and cheating.
Transport commissioner Bhaskar Rao at Yavanika said: “People are beginning to hate auto drivers’ wayward ways.”

Padmasree H Sundaram of Easy Auto said the latest campaign aims to ensure a safer commute.
“There have been a few isolated incidents, but they are disturbing,” she said, adding that there are good and bad sides to travelling by an autorickshaw.
“There are drivers like Auto Shankar and Auto Raja. One returned jewellery worth Rs 8 lakh, the other runs an old-age home.

At the same time, I am told there are some bad drivers operating on Old Madras Road, towards K R Puram and Whitefield.. .
They overcharge and don’t allow the good guys to operate. I was told there is a similar mafia in Electronic City,” she said.

Just call 9844112233 any time you want an auto to commute and we will send it to your doorstep.
Our auto rickshaws have GPS devices installed in them, and we track our vehicles using highly sophisticated servers that track the satellite data from the GPS devices that is updated every 3 minutes.
Our autos are safe, our drivers are always polite and at your service. (Source: http://www.easyauto .in/ )

Check the link for more info:
http://www.easyauto .in/

Compression of unique values

As I indicated in previous article if for every 256 bytes each value appears only ones, then we can achieve the compression using any of the below techniques.

1. Using insertion sort – by remembering the position.
Using this technique for every 256 byte – we can save exactly 32 bytes, i.e for first element no need to remember any position, for next value one bit sufficient to remember weather current element inserted after or before previous number, for next two numbers we need two bits for each value, for next 4 numbers we need 3 bits for each number and so on until for last 128 values we need 8 bit for each value. It is constant save – i.e for every possible combination we will have 1792 bits or 224bytes. (We save 256 – 224 = 32bytes)

2. Using factorial of 256
In this approach, all possible variations can represented using factorial of 256, that will end up with 1684bits for every possible combination. It is constant save – i.e for every possible combination we will have 1684 bits or 210.5 bytes. (We save 256 – 210.5 = 45.5bytes)

3. Using reverse merge sort – Using this approach use a merge sort and store bit information of the list from where we picked the smaller number. Advantage of this technique is, when one list gets over, we don’t require to store or remember unsorting information for remaining element in other list, we just need to append it as illustrated below.

In best case scenario(when all elements are in ascending order) we just need (1/2*8bit) x 256 = 128 bytes for total 256 input bytes, in worst case we need 1793 bits as illustrated below.

Depending on the unique numbers appear in the list we can save between 128 byte(best case) to 31.75 byte (worst case)

Similarly we can use reverse quick sort technique, however merge sort is better than quick in storing minimal unsorting information, because in merge when one list gets over, we don’t require unsorting information for remaining elements in other list, this feature missing if we use quick sort, but still reverse quick sort can be used with lesser than 2048bits.

How to use above theory when all numbers are not unique?

Well above said theory works when all numbers are unique, but in real time such scenario is rare, instead there will be some duplicates and some unique numbers missing.

In fact random digits created by the RAND group contains on an average 100bytes duplicates(repeated) in every block of 256 numbers. in such data we need to patch the dataset by removing non unique, and inserting missing unique numbers, so that above reverse merge sort can be applied. When we remove duplicates we need to retain its position and actual value. (No need to retain the introduced missing numbers, which will be discarded after unsorting. But to patch this way we need lot of byte, e.g: if 100 numbers is duplicates (i.e 100 numbers are missing), we need to have 100*2=200 bytes. (very costly).

However there is better patch as below
1. Read 256 block of data, mark the duplicates/non unique, remember these relative to current position of the number (so we don’t require 8bits for position), Now we have missing unique numbers.
2. Rearrange the input data by eliminating duplicates so that unique numbers moves upper part of the list, fill missing unique numbers at the end of list. (make sure all missing numbers are filled in ascending order so that we can avoid storing unsorting information of missing numbers – which is of nu use for us – This way we can save few bytes)
3. Perform merge sort on the list excluding introduced missing numbers, now the compressed file will have Unsorting information + duplicate number positions + duplicate values.
4. If missing numbers are less than 32 byte we can actually store position using eight bits per position, if it is more than 32, then we use a bit for each byte indicating duplicate or not.

Using above technique I developed a compressor which can compress random data up to 42 duplicates or missing unique numbers. (I used million digit of Rand with some modification to maintain maximum 42 missing numbers in every 256 input bytes. However I am yet to find way to address another 60 bytes so that million digit of Rand group can be compressed at least by a byte.

I will fine tune my compressor and announce in next article in next year. (After Xmas/new year), next article I will introduce another technique of transition or state representation for random data.

Wish you all happy new year 2010.

The answer is NO (…. and yes!!!)

This is little lengthy blog to show and prove the difficulties and possible ways. This article divided into three section.

(Before you read further I suggest you to read Mark Nelson “The Million Random Digit Challenge“)

1. Why it is not possible to compress random data. (brief introduction to Kolmogorov theory)

2. Why it is possible (Quantum theory, matter and anti matter introduction)

3. The future for compression and possible solution with transition representation and unsorting techniques

If you want to skip this introduction and continue with next article, please click – using unsorting techniques for random data compression.

1. Why it is not possible to compress random data. (brief introduction to Kolmogorov theory)

As per pigeonhole principle – if n pigeons are put into m pigeonholes with n > m, then at least one pigeonhole must contain more than one pigeon or if n<m there should be at least one empty pigeonhole. In pure mathematics term – There does not exist an injective function on finite sets whose codomain is smaller than its domain

The Russian mathematician Kolmogorov proved thru his complexity theory that – With the uniform probability distribution on the space of bitstrings of length n, the probability that a string is incompressible by c is at least 1 − 2^(-c+1) + 2^−n (More details please refer to wikipedia here)

Lets take a arithmetic approach to prove the above theory.

The byte i.e 8 bit can represent 256 variety of data i.e 0, 1, 2, ….255 – As you can see there are totally 256 variation or possible values and assume if each one appears only once(in 256 byte) then you cannot represent all these possibilities with a domain contains smaller set than 256.

All existing compression algorithms can compress the data if a data set where all possibilities are not present. e.g: In a input data if only 8 unique bytes present, then we just need 3 bits(0, 1, 2, ….7 – total possibilities) to represent each byte of the input data.

Note: Each algorithm works differently, however all existing algorithms like dictionary based, LZW, Huffman, RLE etc works based on patterns, repetitions, appearance, occurrence or uniqueness of the data.

People often fail to digest the simplicity of the above theory and often claim they have developed magic data compressor which can compress any type of data. (Refer comp.compression)

(Note: I am not claiming any such magic compressor – but I claim such possibilities)

2. Why do I believe that random data compression is possible?

Don’t mistake me, I strongly believe in Kolmogorov complexity and number theory, It is impossible to develop magic compressor which can compress any type of DATA.

In my school days I learnt that our universe was in the form of bindurupi (in the form of dot or originated from nothing) and expanded (still expanding), generating so many possibilities and varieties. As per the quantum theory matter and anti matter joins together – during this process new variations appears. Better example is each couple produce totally different baby, the new baby is neither 100% of father or 100% of mother (or not even 50% father + 50% mother), Baby might be having few feature of father and mother and ancestors, but it also got its own uniqueness. Genes keep mutating, but based on what?

Note: “Nothing” doesn’t mean empty or zero, we are still not clear about nothing. In Sanskrit and Sanathan mythology it is mentioned about Brahma vidya which deals with separation of matter and anti matter.

A matter is a materialistic world or physical world which we can see, we can feel or we can represent, and store (Random data is also a type of matter), whereas as anti matter we cannot imagine or can be represented, even quantum theory couldn’t provide any visual way to see or feel this. (May be human beings are not yet ready to digest this because we are still living in materialistic world)

As per Einstein’s Energy mass relationship E = mc^2 which states “E” energy generated when a object of mass “m” is converted to energy, neither mass nor energy were conserved separately. This is basic used in Nuclear fusion.

Now you might be thinking – What is the relation between Einstein’s mass/energy theory and random data compression?

In one of the lecture Einstein mentioned that you can convert mass “m” to energy “E”, neither mass or energy is destroyable. Similarly you can convert energy to matter !

One of the audience asked Einstein how is it possible to generate mass from energy? – effectively if you have enough energy you can generate any type of object like Gold, Uranium etc.

Einstein said it is outside the definition of Physics and a new world opens thru spiritual way to prove it. Mostly Einstein had a idea of anti matter at that time. In my view anti matter is huge energy reservoir which acted on matter to generate variety of materials we see today. (Imagine Nuclear fission of heavier material), please note I believe there should different type of antimatter similar to matter we see)

We heard many such information in ancient India and even current time it is believed that few got such capability to generate objects. e.g:

1. Pandavas had Akshya patre – to generate food
2. Kamadhenu – A holy cow could provide anything sage Vasista asked for
3. Devine Sri Sathya Sai Baba

I will not go further in spiritual way, which I cannot address properly or I have no sufficient knowledge in this area.

As long as we treat the material as data we cannot compress the random data, the day we find the way to represent the anti matter we can achieve this or in other words we should deal with existing random data as transition or state representation, because data is static form of Bhrahma vidya, it is anti matter which generates all variations.

What does it mean transition or state representation?

There exist a huge set of static data – The size of static data is beyond the imagination of human being. The so called anti matter acts on these matter and generates new set of static data. (This is how universe is expanding?)

The static data is growing! then how come it is static? (I used static to represent default or initial set) During this process there is a different stage of the matter it passes thru. e.g: Radio activity material decays over a period of long time to generate another material, during this long process in the intervals we have variety of data or material, if we know the state of the item we can accurately tell the process involved in it or time involved in it, Similarly if we knew all the process it passed thru, then we can define the state of the material.

Kolmogorov turing machine is actually generating data from above said static data. We call it as highly random or high entropy because we cannot define it as set or within domain. We human beings not reached the stage where we can define the random. Note: Random doesn’t mean uncertainty or non guessable future, but it is predefined beyond the imagination or understanding limit of materialistic human.

Summary : As long as we treat the random numbers as data we cannot compress it, because number theory which applicable to data clearly proves that it is impossible. However the day if we find the way to treat them as either anti matter or state transition then we can change the way we represent the data.

3. The future for compression and possible solution with transition representation and unsorting techniques

In above section I mentioned about state or transition representation, now if we represent random data as state or transition then we will see totally different scenario or concept.

I will take a small portion of the static data I mentioned above, i.e a subset of 256 unique values. That mean I need 8 bits to represent all variations. If in every 256 data each value appears only once then none of the existing algorithm can compress the data. (Although such entropy chances in real time is not high, but they are highly random that cannot be compressed)

[I will try to prove this using conventional manner instead of mathematical form, I don't want my readers get confused with mathematical formulas, I will try to avoid any use of formula or function unless it is un-avoidable].

Using a unsorting algorithm we can achieve compression of such data between 10% to 50%!!!.

The catch here is data set is fixed, but only their positions are different or data not organized. When all numbers are unique we need to find a way to represent original position (Which requires lesser than 8 bits per value)

I hope I have not confused you, let me describe in other way

I have a set of 256 variations and all are unique – In this scenario I need 8bits to represent each value, The values might be 250, 140, 0, 130, 239, … and so on, each value appears only once.

If you use any of the existing compression algorithm we cannot achieve any compression, since there is no repetition or patterns present (Highly random and each value is unique)

In fact most of the compressor generates compressed data in the form of above mentioned form i.e highly unique and non repeatable patterns (Close to unique within every 256 block, but not exact) A better example is random digits created by the RAND group. Please refer Mark Nelson The Million Random Digit Challenge Revisited. (This data is very close to each 256 variations, but not 100% unique – that is based on Kolmogorov complexity)

But my methods can compress such random data (Not Kolmogorov complexity or his Turing machine based random data – because it is not 100% unique within 256 block, I am yet to reach there)

What is unsorting algorithm? In our graduation or engineering we studied various sorting algorithms like bubble sort, quick sort, merge sort etc, However we never came across anything called unsorting algorithm! – Why we need it?

Actually an unsorting algorithm is shuffling back the sorted items back into original order or position. Although such algorithms may not be that useful in real time applications, but I will demonstrate compression of 256 unique data using that.

If there are 256 data, then there are 256 positions that means still I need 8 bit to store each position. However unsorting algorithm is not just about placing the sorted order into original list, it also uses minimal place or storage required to represent original places.

Lets take merge sort, if we have list of two items, then we just need 1 bit(both best and worst case) to unsort back to original list, that bit represent which is smaller among two. If we have four items then we need 4 bits best case scenario and 5 bits in worst case, If we have 8 items then we need 12 bits best case and 17 bits worst case scenario. (Note: In case of 8 items we need actually 24 bits to represent all unique values), If we have 256 unique items we need 1024bits(128byte) best case and 1856 worst case. Actually 256 unique value requires 2048 bits. As you might observed using reverse merge sort we can achieve the compression mentioned above. I will not go in details, because it will fill another page. If you need further details please contact me. Please write me if you need a working code of such compression using reverse merge sort. Similar results can be obtained using reverse quick sort.

Above compressor works well for each value uniqueness within every 256 values, However in real time data is not that unique and Kolmogorov doesn’t guarantee that entropy within predefined set of data.

The above concept is same as probability theory, i.e If we toss a coin chances of getting either head or tail is 50%, However it won’t guarantee that for every two successive toss one will head and other will be tail. But over large number of toss you will end up with 50% head and 50% tail.

Coming back to random digits created by the RAND group – I observed for every 256 block of data uniqueness of values lies anywhere between 166 to 146, that means if we find a way to patch the missing unique number within every 256 block, then we reached the stage of random data compression, however patched unique number + unsorting information should be less than 2047 bits for every 256 block. I am still looking for such possibility.

Well now other approach, as I mentioned in previous section instead of representing random data as data, but as state or transition then new concept opens. i.e treat random digits of RAND group as unique transition of another process.

Continue reading next article – using unsorting techniques for random data compression.

There was a farmer who grew superior quality, award-winning corn in his
farm. Each year, he entered his corn in the state fair where it won
honors. & Prizes.

One year a newspaper reporter interviewed him and learned something
interesting about how he grew his corn. The reporter discovered that the
farmer shared his seed corn with his neighbors.

“How can you afford to share your best seed corn with your neighbors when
they are entering corn in competition with yours each year?” The reporter
asked. “Why brother”

” The farmer replied, “Didn’t you know? The wind picks up pollen grains
from the ripening corn and swirls it from field to field. If my neighbors
grow inferior, sub-standard & poor quality corn, cross-pollination will
steadily degrade the quality of my corn. If I have to grow good corn, I
must help my neighbors to grow good corns.”
The farmer gave a superb insight into the connectedness of life. His corn
cannot improve unless his neighbors’ corn also improves. So it is in the
other dimensions and areas of life!
Those who choose to be in harmony must help their neighbors and colleagues
to be at peace. Those who choose to live well must help others live well.
The value of a life is measured by the lives it touches…
Success does not happen in isolation; it is most often a participatory and
collective process. So share the good practices, ideas and new knowledge
with your family, friends, team members and neighbors & all. As they say:
“Success breeds Success.”