Mailing List Archive

xen with huffman coding
Hello all,
Is it possible to integrate Xen pre-copy algorithm with
huffman coding compression algorithm. If during live migration of vm pages
are first compressed and then transferred at destination.

regards,
DigvijaySingh
Re: xen with huffman coding [ In reply to ]
On 14/12/12 14:32, digvijay chauhan wrote:
> Hello all,
> Is it possible to integrate Xen pre-copy algorithm with
> huffman coding compression algorithm. If during live migration of vm
> pages are first compressed and then transferred at destination.
Short answer:
Probably.

Longer answer:
Have a look at the code, do some experimental changes, and see how much
it improves. The question is if it's sufficiently more efficient to
compress and then transfer the package, then uncompress it, vs. the
uncompressed version. The answer to that depends, partly, on what the
network setup is on the system, and how well the huffman coding is done.

For code and typical data, I'm not at all convinced that huffman
encoding (which is based on run-lengths) is the best method. You may
want to look at byte-pair encoding instead, which is more suitable for
"stuff that contains byte values" - I believe it is also possible to do
"in situ", meaning the encoding never takes more space than the original
data - but I could have got that wrong.

Looking at different compression methods would be a good research project.

What is your goal (e.g. is downtime or overall transfer time the
important factor)?
Generally, downtime is not very much affected by the amount of time it
takes to copy the guest, except for the final iteration, which is a
small portion of the total copy time. Of course, the total copy time for
a large guest could easily be several minutes if the guest is also
making lots of pages dirty.

What is your test setup?

What kind of network hardware are you using?

--
Mats
>
> regards,
> DigvijaySingh


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
Re: xen with huffman coding [ In reply to ]
On 14/12/2012 14:49, "Mats Petersson" <mats.petersson@citrix.com> wrote:

> For code and typical data, I'm not at all convinced that huffman
> encoding (which is based on run-lengths) is the best method.

Actually Huffman encoding is not a run-length scheme.

-- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
Re: xen with huffman coding [ In reply to ]
On 14/12/12 16:27, Keir Fraser wrote:
> On 14/12/2012 14:49, "Mats Petersson" <mats.petersson@citrix.com> wrote:
>
>> For code and typical data, I'm not at all convinced that huffman
>> encoding (which is based on run-lengths) is the best method.
> Actually Huffman encoding is not a run-length scheme.
Ah, I'm confusing it with ccitt (or whatever it is that fax-machines
use), which uses a fixed Huffman tree to encode a set of run lengths of
black/white pixels.

Either way, looking at more than one compression mechanism may have some
value.

--
Mats
>
> -- Keir
>
>
>
>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
Re: xen with huffman coding [ In reply to ]
On 14/12/2012 16:34, "Mats Petersson" <mats.petersson@citrix.com> wrote:

> On 14/12/12 16:27, Keir Fraser wrote:
>> On 14/12/2012 14:49, "Mats Petersson" <mats.petersson@citrix.com> wrote:
>>
>>> For code and typical data, I'm not at all convinced that huffman
>>> encoding (which is based on run-lengths) is the best method.
>> Actually Huffman encoding is not a run-length scheme.
> Ah, I'm confusing it with ccitt (or whatever it is that fax-machines
> use), which uses a fixed Huffman tree to encode a set of run lengths of
> black/white pixels.
>
> Either way, looking at more than one compression mechanism may have some
> value.

Yes it was a minor point really, and noone really uses Huffman on its own
afaik. More broadly I agree -- it looks like an interesting student project.

> --
> Mats
>>
>> -- Keir
>>
>>
>>
>>
>



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel