Comments on: Random data compression – Is it possible? How to use merge sort? http://blog.adityon.com/2010/06/random-data-compression-is-it-possible-how-to-use-merge-sort/ Simpler solution for complex problem. Think different - Keshav Shetty Thu, 19 Jan 2012 10:16:22 +0000 hourly 1 http://wordpress.org/?v=3.3 By: Keshav Shetty http://blog.adityon.com/2010/06/random-data-compression-is-it-possible-how-to-use-merge-sort/comment-page-1/#comment-565 Keshav Shetty Thu, 16 Jun 2011 08:50:22 +0000 http://blog.adityon.com/2010/06/random-data-compression-is-it-possible-how-to-use-merge-sort/#comment-565 Ibrahim, For more details, You can check the comment here - http://blog.adityon.com/2010/10/random-data-compression-one-bit-diff-encoding-continued/comment-page-1/#comment-543 Ibrahim,
For more details, You can check the comment here – http://blog.adityon.com/2010/10/random-data-compression-one-bit-diff-encoding-continued/comment-page-1/#comment-543

]]> By: Keshav Shetty http://blog.adityon.com/2010/06/random-data-compression-is-it-possible-how-to-use-merge-sort/comment-page-1/#comment-564 Keshav Shetty Thu, 16 Jun 2011 08:47:28 +0000 http://blog.adityon.com/2010/06/random-data-compression-is-it-possible-how-to-use-merge-sort/#comment-564 Hi Ibrahim, with a bit we just know or marked duplicates, but we don't have information of actual shuffling of non unique numbers. In order re generate original shuffling order we need to use sorting/unsorting technique. Please check the code sample I posted in my merge sort article. Hi Ibrahim,
with a bit we just know or marked duplicates, but we don’t have information of actual shuffling of non unique numbers.
In order re generate original shuffling order we need to use sorting/unsorting technique.
Please check the code sample I posted in my merge sort article.

]]> By: Ibrahim http://blog.adityon.com/2010/06/random-data-compression-is-it-possible-how-to-use-merge-sort/comment-page-1/#comment-550 Ibrahim Tue, 07 Jun 2011 22:53:07 +0000 http://blog.adityon.com/2010/06/random-data-compression-is-it-possible-how-to-use-merge-sort/#comment-550 Why is this sort necessary when we already have marked the duplicates? Sorry but I did not understand the purpose of thus sort, as far I see the input is just shuffled, I have confusion may be is that you are finding the missing values after this sort? bye Ibrahim Why is this sort necessary when we already have marked the duplicates? Sorry but I did not understand the purpose of thus sort, as far I see the input is just shuffled, I have confusion may be is that you are finding the missing values after this sort?

bye

Ibrahim

]]> By: Adityon » Blog Archive » Random data compression – One bit diff encoding continued - Blog by Keshav Shetty http://blog.adityon.com/2010/06/random-data-compression-is-it-possible-how-to-use-merge-sort/comment-page-1/#comment-422 Adityon » Blog Archive » Random data compression – One bit diff encoding continued - Blog by Keshav Shetty Fri, 01 Oct 2010 18:03:41 +0000 http://blog.adityon.com/2010/06/random-data-compression-is-it-possible-how-to-use-merge-sort/#comment-422 [...] 5. Apply merge sort and store bit information of sorting data. (Refer how to use merge sort article) [...] [...] 5. Apply merge sort and store bit information of sorting data. (Refer how to use merge sort article) [...]

]]> By: Ernst Berg http://blog.adityon.com/2010/06/random-data-compression-is-it-possible-how-to-use-merge-sort/comment-page-1/#comment-389 Ernst Berg Tue, 29 Jun 2010 18:39:50 +0000 http://blog.adityon.com/2010/06/random-data-compression-is-it-possible-how-to-use-merge-sort/#comment-389 I enjoy that folks are thinking on this. I am one of those who have been inspired to try this "Million-Digit challenge and I too have good ideas. I have something to chat about this year. My take on compressing the Milliondigit.bin file is that what people have been saying is true; there isn't any exploit as they knew it. So I decided to explore expressing the file in different ways with systems I constructed and I have worked at it at a hobbyist's pace for many years. This has brought me to "one to one bijective transforms" this year. This last January I discovered that all binary strings can be cycled through all other binary strings bijectively of that length on a one to one level. Example: Given some 8 bit long string there are 255 strings it will cycle through and will become that original object again. Now the processing time goes up the longer the string so length of Million-Digit file is 3321928 bits long so the cycle is 2^3321928 strings or 9.363453492e+999999 strings if I have my math straight in my head today.. LOL I realize I posted some silly numbers the other day.. LOL I am a hobbyist so I guess I can be forgiven. Needless to say it takes an extremely long time per step/transform on a 2ghz core and there are a huge number of strings to step through. Cycling through all the strings of milliondigit.bin length on a 2 GHZ core takes forever.. How would it compress? I am thinking the distance from our source string to some pattern string we can assume is a number we can represent with smaller data. That is how I see this bijective transform "compressing." How can we lose if any string can be any other string and back with a simple index? I hope I am being clear.. I make silly mistakes too often LOL. I am at the point now that I could benefit from a mentor/friend.. That and access to a fast processor. Do they make 100ghz processors? Thanks for the forum.. Ernst_Berg@sbcglobal.net I enjoy that folks are thinking on this. I am one of those who have been inspired to try this “Million-Digit challenge and I too have good ideas.
I have something to chat about this year.

My take on compressing the Milliondigit.bin file is that what people have been saying is true; there isn’t any exploit as they knew it. So I decided to explore expressing the file in different ways with systems I constructed and I have worked at it at a hobbyist’s pace for many years. This has brought me to “one to one bijective transforms” this year.

This last January I discovered that all binary strings can be cycled through all other binary strings bijectively of that length on a one to one level.

Example: Given some 8 bit long string there are 255 strings it will cycle through and will become that original object again.

Now the processing time goes up the longer the string so length of Million-Digit file is 3321928 bits long so the cycle is 2^3321928 strings or 9.363453492e+999999 strings if I have my math straight in my head today.. LOL I realize I posted some silly numbers the other day.. LOL I am a hobbyist so I guess I can be forgiven. Needless to say it takes an extremely long time per step/transform on a 2ghz core and there are a huge number of strings to step through.
Cycling through all the strings of milliondigit.bin length on a 2 GHZ core takes forever..

How would it compress?

I am thinking the distance from our source string to some pattern string we can assume is a number we can represent with smaller data.
That is how I see this bijective transform “compressing.”
How can we lose if any string can be any other string and back with a simple index?

I hope I am being clear.. I make silly mistakes too often LOL.

I am at the point now that I could benefit from a mentor/friend..
That and access to a fast processor. Do they make 100ghz processors?

Thanks for the forum..

Ernst_Berg@sbcglobal.net

]]> By: Keshav Shetty http://blog.adityon.com/2010/06/random-data-compression-is-it-possible-how-to-use-merge-sort/comment-page-1/#comment-350 Keshav Shetty Tue, 08 Jun 2010 06:44:00 +0000 http://blog.adityon.com/2010/06/random-data-compression-is-it-possible-how-to-use-merge-sort/#comment-350 Hi Ibrahim, Thanks for visiting my blog, I guess you have not read other article I mentioned. Have a look at http://blog.adityon.com/category/data-compression Million random digit is not yet compressible. The reason is million random digit contains approximately 90-110 bytes of duplicates(doubles) for every block of 256bytes, where as my compression program works up to 42 duplicates within every block of 256bytes. I am sharing my finding so that others may comment and discuss their ideas, so that we can reach from 42 to 110 doubles. Thanks & regards Keshav K Shetty Hi Ibrahim,

Thanks for visiting my blog, I guess you have not read other article I mentioned. Have a look at http://blog.adityon.com/category/data-compression

Million random digit is not yet compressible. The reason is million random digit contains approximately 90-110 bytes of duplicates(doubles) for every block of 256bytes, where as my compression program works up to 42 duplicates within every block of 256bytes.
I am sharing my finding so that others may comment and discuss their ideas, so that we can reach from 42 to 110 doubles.

Thanks & regards
Keshav K Shetty

]]> By: Ibrahim http://blog.adityon.com/2010/06/random-data-compression-is-it-possible-how-to-use-merge-sort/comment-page-1/#comment-346 Ibrahim Mon, 07 Jun 2010 18:51:15 +0000 http://blog.adityon.com/2010/06/random-data-compression-is-it-possible-how-to-use-merge-sort/#comment-346 Please post some about your tests on AMilliondigit bin file by Matt Mahoney and Mark Nelson. They said even single byte cannot be reduced. bye Please post some about your tests on AMilliondigit bin file by Matt Mahoney and Mark Nelson. They said even single byte cannot be reduced.

bye

]]>