Lzma slow when compressing streams or byte[]

Jul 10, 2009 at 12:15 AM
Edited Jul 10, 2009 at 12:16 AM

Hi,

I'm using SevenZipSharp to compress serialized objects. I've tried this approach:

internal static byte[] Serialize(object element)
{
    using (MemoryStream ms = new MemoryStream())
    {
        BinaryFormatter formatter = new BinaryFormatter();
        formatter.Serialize(ms, element);

        return SevenZipCompressor.CompressBytes(ms.ToArray());
    }
}
And this one:
internal static byte[] Serialize(object element)
{
    using (MemoryStream ms = new MemoryStream())
    {
        using (LzmaEncodeStream zip = new LzmaEncodeStream(ms))
        {
            BinaryFormatter formatter = new BinaryFormatter();
            formatter.Serialize(zip, element);
        }

        return ms.ToArray();
    }
}
I'm serializing several lists of doubles and ints (between 250.000 and 2.000.000 elements each) and both approaches take around 100 seconds to serialize all the content (I was using GZipStream before and it was taking like 7 seconds).
The compression rates of both approaches are far better than the ones of the .NET GZipStream, but i was surprised about the time difference, 7zip is pretty fast when zipping files so I was surprised to see it take so much time zipping those serialized objects. Am I doing something wrong?
Also I haven't found any way of changing the compression level when using those approaches, is there something I'm not taking into account?
Regards,
Vicente
Coordinator
Jul 10, 2009 at 7:00 AM

Vicente, there is a way to change the compression level by changing the LZMA dictionary size. It is either a property of LzmaEncodeStream or a static property of Compressor (can't write the exact name because haven't got a computer in the country). The less dictionary size, the faster compression but the bigger size.
The poor perfomance you report about is very interesting. You use classes properly. You wrote about zipping but CompressBytes and LzmaEncodeStream perform lzma-ing (compressing with LZMA algorithm) and not zipping, hence the speed decrease and good compression ratio. The managed LZMA implementation was taken directly from 7zip C#, i consider to improve it on my summer holidays. The funny thing is that some guys from channel 9 singed songs about the speed of managed LZMA and this seems to be far from true.