4
Vote

System.Collections.Generic.KeyNotFoundException on CompressFiles

description

When executing the CompressFiles function I had the following exception:
 
System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
at SevenZip.SevenZipLibraryManager.OutArchive(OutArchiveFormat format, Object user)
at SevenZip.SevenZipCompressor.CompressFilesEncrypted(Stream archiveStream, Int32 commonRootLength, String password, String[] fileFullNames)
at SevenZip.SevenZipCompressor.CompressFilesEncrypted(String archiveName, Int32 commonRootLength, String password, String[] fileFullNames)
at SevenZip.SevenZipCompressor.CompressFilesEncrypted(String archiveName, String password, String[] fileFullNames)
at SevenZip.SevenZipCompressor.CompressFiles(String archiveName, String[] fileFullNames)
 
7zip version: 0.64.3890.29348
Happened in multi-threaded environment, however, since I always create new SevenZipCompressor before a compression action, it shouldn't matter. Doesn't reproduce easily, happened only once so far.
 
Thanks!

file attachments

comments

jeffreypry wrote Nov 16, 2011 at 5:44 PM

I am having the same issue! It's driving me crazy. I can't reproduce it easily. It happens every 100 or so calls and occurs in a multi-threaded environment as well.

wrote Nov 16, 2011 at 5:44 PM

jeffreypry wrote Nov 16, 2011 at 5:45 PM

Here is the stacktrace from the error:

System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
at SevenZip.SevenZipLibraryManager.InitUserOutFormat(Object user, OutArchiveFormat format)
at SevenZip.SevenZipLibraryManager.LoadLibrary(Object user, Enum format)
at SevenZip.SevenZipCompressor.CompressStreamDictionary(Dictionary2 streamDictionary, Stream archiveStream, String password)
at SevenZip.SevenZipCompressor.CompressStreamDictionary(Dictionary
2 streamDictionary, String archiveName, String password)
at EDWFileController.Loader.LoadController.ProcessOneArchiveFile(ArchiveFile archiveFile)
at System.Threading.Tasks.Task.Execute()

jeffreypry wrote Nov 16, 2011 at 8:52 PM

Looks like a static lock object in the class around the compression logic did the trick. Try it out.... Can we get this built into the next release? Maybe a global Mutex or solve the issue entirely? Thanks!

wrote Dec 1, 2011 at 12:50 PM

JohnBonano wrote Dec 15, 2011 at 7:37 PM

I am having the same issue too Jeffrey.

wrote Dec 21, 2011 at 7:19 PM

scottlerch wrote Dec 21, 2011 at 10:41 PM

Since LibraryManager is static every method should be thread safe. I would recommend creating a readonly static object syncRoot field and locking each public method on it. Locking on _libraryFileName as is currently done isn't safe since the public SetLibraryPath method can modify it. Even after making those changes though I still got some COM exceptions due to some other race condition or perhaps one thread tried using a COM object from another thread, but I couldn't easily track down the problem. Either way there are major threading issues with LibraryManager. The following issues all seem to be related to the same problems: 9786, 8486, 6080. To recreate the issue you may need a machine with lots of hardware threads, mine has 8 and I've found many a threading issue in various open source projects that never tested on such a machine.

scottlerch wrote Dec 21, 2011 at 10:43 PM

I can recreate this problem every time using several threads compressing various files at once on a machine with 8 hardware threads. Same issue as 8486, 6080. See my comments in 9786.

wrote Dec 21, 2011 at 11:29 PM

scottlerch wrote Dec 21, 2011 at 11:32 PM

Attached is a patch that fixes all threading issues. I did a scan for all static fields and made sure they had appropriate locking as needed (e.g. only locked if they had a chance of being modified after static creation) and in one case marked a static int that was only ever assigned volatile. Originally, I missed a few areas that caused some weird COM exceptions, but now everything works. I tested having 8 cores compressing away in parallel on a few thousand files without any issues.

chiaming0914 wrote Jul 7, 2012 at 1:45 AM

Help!!!
How to use ThreadingIssueFix.patch to patch SevenZipSharp.dll

jeffreypry wrote Jul 10, 2012 at 11:29 AM

To use the patch you download or check out a copy of the source code and use something like WinMerge to apply the patch the the files. Pretty simple.

wrote Feb 22, 2013 at 1:16 AM