Sep 17, 2010 at 9:39 PM
Edited Sep 17, 2010 at 11:55 PM
@markhor we are using the lates .64 version against 9.x beta release. The FastCompression flag doesn't really help at all on CompressDirectory when it contains 1000's of files (all small XML files) it is still about ~6x slower than using
7z directly. It seems SevenZipSharp is great for small number of files, but in large numbers it is not useful, it is faster to just shell out and use the command line version.
Is this some form of COM overhead? I would think as the SevenZipSharp is a wrapper so it just has to pass off the call to the DLL once and then the 7z.dll is doing all the work, what could be causing this to be this slow?
10,863 files in 13 folders totaling 320MB
Settings same between 7z native and SevenZipSharp
LZMA, Ultra, AES-256, EncryptHeaders = True, DirectoryStructure = true, FastCompression = true
SevenZipSharp took 6 minutes and 8 seconds
Native 7z using GUI took 1 minute and 17 seconds.
Output was identical between the two otherwise.
Looking through the SevenZipCompressor source I noticed that when calling the CompressDirectory that it converts the directory into a List<string> of the files in the directory and then calls CompressFilesEncrypted. This then converts that List<string>
to a FileInfo, before calling into the low-level wrapper.
I do not know what the 7z.dll exposes, however it would seem that it may expose a way to just pass the directory provided like you can at the EXE or in the GUI, and then let the native code deal with all the iteration of the files. Not even
sure if this is any part of the impact.
I also noticed that the default Dictionary size in 7z is 64 MB and word size is 64, and solid block size is 4 GB, it also has an option of number of CPU threads with a default of 2 out of 8.
I change your default SevenZipCompressor.LzmaDictionarySize from 1 << 22 to 1 << 48 to set it to 64 MB. I couldn't find where you are setting the word size and solid block size, but the dictionary size didn't impact performance at
Is it possible to provide settings in the SevenZipCompressor to control the word and solid block size, and even if we can control the number of CPU threads it will use. I know that you are limited to what is available in the 7z.dll, so I would
understand if these options are not available for some reason.