2
Vote

System.IndexOutOfRangeException during CompressStream

description

I've been using SevenZipSharp library fine for quite some time, but today I got the same error twice only minutes apart.
This is the stack trace:
System.IndexOutOfRangeException: Index was outside the bounds of the array.
at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
at SevenZip.SevenZipLibraryManager.InitUserOutFormat(Object user, OutArchiveFormat format)
at SevenZip.SevenZipLibraryManager.LoadLibrary(Object user, Enum format)
at SevenZip.SevenZipCompressor.CompressStream(Stream inStream, Stream outStream, String password)
at SunLineWcfService.SynchronizationService.SerializeAndCompress(Object& graph, Boolean useLzma2) in C:\Users\Mõmmi\Documents\Visual Studio 2010\Projects\SunLineWcfService\SunLineWcfService\SynchronizationService.cs:line 892
 
My SerializeAndCompress method is defined as follows:
private static MemoryStream SerializeAndCompress(ref object graph, bool useLzma2)
{
using (MemoryStream ms = new MemoryStream())
{
    DataContractSerializer dcs = new DataContractSerializer(graph.GetType());
    dcs.WriteObject(ms, graph);
    ms.Position = 0;
 
    MemoryStream compressedStream = new MemoryStream();
 
    SevenZip.SevenZipCompressor compressor = new SevenZip.SevenZipCompressor();
    if (useLzma2)
    {
        compressor.CompressionMethod = SevenZip.CompressionMethod.Lzma2;
        compressor.CompressionLevel = SevenZip.CompressionLevel.Low;
    }
    else
    {
        compressor.CompressionMethod = SevenZip.CompressionMethod.Ppmd;
        compressor.CompressionLevel = SevenZip.CompressionLevel.Normal;
    }
 
    compressor.CompressStream(ms, compressedStream);
 
    compressedStream.Position = 0;
 
    return compressedStream;
}
}
 
 
This error happened on a Windows Server 2003 Standard R2 x64 Edition, with 7zip 9.20 x64, SevenZipSharp library version 0.64, using .NET4, a WCF application/service running in x64 mode.
The server has fully buffered memory, so I hardly doubt this could have been a memory issue.
 
Could the error have happened due to concurrency?
My SerializeAndCompress method can be called multiple times from different clients, each getting a new instance of my application. I myself don't syncrhonize access to SerializeAndCompress.

comments

markoweb2 wrote Jan 8, 2011 at 8:28 AM

Forgot to add, that both calls, when the error occured, had useLzma2 = false, meaning that the Ppmd compression method was used.

wrote Jan 8, 2011 at 8:31 AM

markoweb2 wrote Jan 14, 2011 at 4:40 PM

Recently got two errors again from within SevenZip.SevenZipLibraryManager.LoadLibrary:
1) System.ArgumentNullException: Value cannot be null.
Parameter name: ptr
at System.Runtime.InteropServices.Marshal.GetDelegateForFunctionPointer(IntPtr ptr, Type t)
at SevenZip.SevenZipLibraryManager.OutArchive(OutArchiveFormat format, Object user)
at SevenZip.SevenZipCompressor.SetCompressionProperties()
at SevenZip.SevenZipCompressor.GetArchiveUpdateCallback(Stream inStream, String password)
at SevenZip.SevenZipCompressor.CompressStream(Stream inStream, Stream outStream, String password)
at SunLineWcfService.SynchronizationService.SerializeAndCompress(Object& graph, Boolean useLzma2) in C:\Users\Mõmmi\Documents\Visual Studio 2010\Projects\SunLineWcfService\SunLineWcfService\SynchronizationService.cs:line 893

2) System.NullReferenceException: Object reference not set to an instance of an object.
at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
at SevenZip.SevenZipLibraryManager.InitUserOutFormat(Object user, OutArchiveFormat format)
at SevenZip.SevenZipLibraryManager.LoadLibrary(Object user, Enum format)
at SevenZip.SevenZipCompressor.CompressStream(Stream inStream, Stream outStream, String password)
at SunLineWcfService.SynchronizationService.SerializeAndCompress(Object& graph, Boolean useLzma2) in C:\Users\Mõmmi\Documents\Visual Studio 2010\Projects\SunLineWcfService\SunLineWcfService\SynchronizationService.cs:line 891


This got me thinking, it can't possibly be a bug inside SevenZipSharp library, because there are three different types of errors already. I ran memtest and of course no memory errors were found, so I looked at my own code more carefully.
First of, I thought there might be bug within Microsoft code. My application is .NET4, but SevenZipSharp is .NET2, maybe the two side-by-side don't work well in a WCF/service enviroment. So I downloaded the source code and rebuilt SevenZipSharp for .NET4.
Secondly, I noticed my SerializeAndCompress method was static. Theoretically it shouldn't be problem, but - the windows service that host's my WCF application is a single life-time instance, it creates new instances of my SynchronizationService class in the same app-domain (presumably) so everything static might be shared between instances. As my SerializeAndCompress method is executed concurrently, maybe there is some type of conflict there. Or if one instance is executing SerializeAndCompress and another instance has just started and is setting the SevenZipBase.SetLibaryPath, maybe those two calls collide...
Anyhow, just to be on the safe side I removed all static keywords, so everything is 100% instance based.

I guess now we will have to wait and see if this has solved the problem. It might take a few weeks/months to be sure. I'll post my results later on...

markoweb2 wrote Feb 11, 2011 at 3:03 PM

The problem seems to be fixed by the last changes I made. No more errors for the past month.
I think the main cause for the problem was my static methods. Perhaps someone can look deeper into this to see whether it's a problem with SevenZipSharp or perhaps static methods in my scenario were simply incorrect programming (in which case this ought to be documented better)..

markoweb2 wrote Feb 14, 2011 at 7:09 AM

Seems I rejoiced to soon. Got another exception today:
System.ArgumentNullException: Value cannot be null.
Parameter name: ptr
at System.Runtime.InteropServices.Marshal.GetDelegateForFunctionPointer(IntPtr ptr, Type t)
at SevenZip.SevenZipLibraryManager.OutArchive(OutArchiveFormat format, Object user) in C:\Users\Mõmmi\Desktop\sevenzipsharp-59007\SevenZip\LibraryManager.cs:line 503
at SevenZip.SevenZipCompressor.SetCompressionProperties() in C:\Users\Mõmmi\Desktop\sevenzipsharp-59007\SevenZip\SevenZipCompressor.cs:line 254
at SevenZip.SevenZipCompressor.GetArchiveUpdateCallback(Stream inStream, String password) in C:\Users\Mõmmi\Desktop\sevenzipsharp-59007\SevenZip\SevenZipCompressor.cs:line 758
at SevenZip.SevenZipCompressor.CompressStream(Stream inStream, Stream outStream, String password) in C:\Users\Mõmmi\Desktop\sevenzipsharp-59007\SevenZip\SevenZipCompressor.cs:line 1798

Could it be that SevenZipSharp or 7zip itself can't handle parallel calls in certain methods? So that if the timing is right, these kinds of exceptions occur?

JohnBonano wrote Dec 15, 2011 at 10:03 PM

So what did you do to resolve your issue? From reading the discussions, it appears that adding a lock around the compression is the answer. I would be curious to know what you did to fix your issue.

Thanks

wrote Jun 15, 2012 at 4:52 AM

wrote Feb 22, 2013 at 1:16 AM