Speed of CHM Publishing

Please post all questions relating to Help & Manual 6 here!

Moderators: Alexander Halser, Tim Green

Marvin Mueller
Posts: 10
Joined: Tue May 31, 2016 9:29 am

Speed of CHM Publishing

Unread post by Marvin Mueller »

Hi board,

we're running H&M 6.5.5.3020 with a help project containing 8681 topic files. We do regular exports with different sets of builds included. One export includes nearly all of these 8681 topic files. This export takes approximately 3-4 hrs. The creation of the .html files in the ~tmpchm directory takes (for one file) from a couple of milliseconds up to approx. 20 seconds, even for topics that do not contain heaps of images or text.

Therefore my question is, whether there are any options available to speed up the chm export process.

Some background info:
  • i3-2100 @ 3.10GHz, Windows 7 SP 1, 16GB RAM
  • Help & Manual on network share; help files & export directory on local disk
  • CPU utilisation: 20-30%, RAM utilisation: 20%, network utilisation: none
Thanks for any advice!

Best regards
Marvin
User avatar
Tim Green
Site Admin
Posts: 23153
Joined: Mon Jun 24, 2002 9:11 am
Location: Bruehl, Germany
Contact:

Re: Speed of CHM Publishing

Unread post by Tim Green »

Hi Marvin,

Note that the current build of version 6 is 6.5.6 Build 3027. You can get the installer here:

http://www.helpandmanual.com/downloads.html#oldver

Even with that many topics, the time this is taking sounds excessive. The most common cause for things like this are network bottlenecks, so even though you say that the source and output are all local the first thing to check would be whether you are getting anything from the network while you are working.

The next thing to check are the entries in your project search path (Configuration > Common Properties > Project Search Path). A well-designed project should never have more than about 10 entries here. In addition to this, it's essential to check that all the entries you are using are valid. Help and manual locates graphics files and snippet files by searching through the folders in this list and inserting the first matching file that it finds. If the list contains folders that do not exist, particularly if they are network folders, then the search for each individual file can take a long time. Windows is incredibly slow and inefficient about resolving references to network folders that it cannot find. In addition to this, just having many folders in the list can also slow down operation if the files that need to be found are in folders that are far down the list, particularly if network folders need to be checked on the way.

Consolidating your graphics files into a small compact number of folders is very easy to do. See this topic in the help for details:

Basic Working Procedures > Using Graphics > Managing your graphics
Regards,
Tim (EC Software Documentation & User Support)

Private support:
Please do not email or PM me with private support requests -- post to the forum directly.
Marvin Mueller
Posts: 10
Joined: Tue May 31, 2016 9:29 am

Re: Speed of CHM Publishing

Unread post by Marvin Mueller »

Hi Tim,

thanks very much for your reply.

The only thing in the help project, that is network based, is the application itself (it's Server Edition). I do not observe excessive network usage in Task Manager during simulation runs. Well, almost none during most of the time.

The project search path contained four dead links, which I removed. There are three paths left, two containing images, one containing snippets. All paths are on a local partition. These dead links were the at the end of the list.

In the last days I converted all images to png and removed illegal chars from image's file names (like ~), as this is also done during the chm export. This speeded up export by approx. 5-10 minutes. The removal of four dead links from the project search path gained only a few minutes, maybe because they were the last ones on the list.

Our sysadmin installed the lastest build, therefore I will start a new run today to see if there are any performance improvements.

Here are some numbers from yesterdays export (running 3:50hrs in total):

Compile time: 2 minutes, 50 seconds
13,671 Topics
235,381 Local links
0 Internet links
3,956 Graphics

Best regards & thanks for the effort
Marvin
Marvin Mueller
Posts: 10
Joined: Tue May 31, 2016 9:29 am

Re: Speed of CHM Publishing

Unread post by Marvin Mueller »

After the first export run with 6.5.6.3027 it seems that updating had no significant impact on export performance, unfortunately.

Best regards
Marvin
User avatar
Tim Green
Site Admin
Posts: 23153
Joined: Mon Jun 24, 2002 9:11 am
Location: Bruehl, Germany
Contact:

Re: Speed of CHM Publishing

Unread post by Tim Green »

Hi Marvin,

Hmm... how is the speed of your hard disk? Also, checking and resolving all those local links will also take time, but I'm not sure whether that alone could account for the very long publish time. :?
Regards,
Tim (EC Software Documentation & User Support)

Private support:
Please do not email or PM me with private support requests -- post to the forum directly.
User avatar
Alexander Halser
EC-Software Support
Posts: 4098
Joined: Mon Jun 24, 2002 7:24 pm
Location: Salzburg, Austria
Contact:

Re: Speed of CHM Publishing

Unread post by Alexander Halser »

Does H&M issue a lot of compiler messages? This and the frequent update of the compiler messages window could slow down the export. But you should see this with the Webhelp format as well, not just with CHM.

I recommend to do a command line compilation and compare the times with the UI version.
Alexander Halser
Senior Software Architect, EC Software GmbH
Marvin Mueller
Posts: 10
Joined: Tue May 31, 2016 9:29 am

Re: Speed of CHM Publishing

Unread post by Marvin Mueller »

Hi Tim & Alexander,

I think I have to clarify the situation a little: The full publishing takes ~4hrs. Compiling the CHM file at the very end of the process takes ~3mins. There are no compiler messages except of the usual report at the end.

It is a Samsung HD204UI inside my machine. There are several reports on the internet about relativly slow writing performance of this disk, especially with small files. Fortunately our sysadmin found an unused Toshiba SSD, that he installed in my machine for a test run last night.
Unfortunately again export took nearly 4 hours to complete.

I will do a command line export with CHM and HTML format this night or during the day, maybe this boosts performance.

Thank you for your help

Marvin
Last edited by Marvin Mueller on Thu Jun 09, 2016 10:49 am, edited 1 time in total.
User avatar
Alexander Halser
EC-Software Support
Posts: 4098
Joined: Mon Jun 24, 2002 7:24 pm
Location: Salzburg, Austria
Contact:

Re: Speed of CHM Publishing

Unread post by Alexander Halser »

If you see a dramatic improvement, it is the UI updates, that slows down the compilation. With dramatic, I mean a reduction from 4 hrs to 15 minutes or so. If the difference is relatively small (command line compilation is always faster), it is the disk that's the bottleneck.
Alexander Halser
Senior Software Architect, EC Software GmbH
Marvin Mueller
Posts: 10
Joined: Tue May 31, 2016 9:29 am

Re: Speed of CHM Publishing

Unread post by Marvin Mueller »

Hi Alexander,

I did a command line CHM export this morning with project, skin and export directory on the new SSD.

- 09:39 Start of script
- 13:30 Start of help compiler to generate .chm from ~tmpchm diretory
- 13:32 Finish

Some statistics from the command line output:
- Compile time: 2 minutes, 14 seconds
- Topics 13,671
- Local Links 235,381
- Internet Links 0
- Graphics 3,957

As all files were on SSD, I think that disk can be ruled out as a bottleneck.

Regards
Marvin
User avatar
Tim Green
Site Admin
Posts: 23153
Joined: Mon Jun 24, 2002 9:11 am
Location: Bruehl, Germany
Contact:

Re: Speed of CHM Publishing

Unread post by Tim Green »

Hi Marvin,

Is it possible that there are a large number of graphics in the project that you have resized in Help & Manual? When you do this the graphics are physically resized during the publishing operation. If there are lots of them then this could be a major contribution to a long publishing operation.
Regards,
Tim (EC Software Documentation & User Support)

Private support:
Please do not email or PM me with private support requests -- post to the forum directly.
User avatar
Alexander Halser
EC-Software Support
Posts: 4098
Joined: Mon Jun 24, 2002 7:24 pm
Location: Salzburg, Austria
Contact:

Re: Speed of CHM Publishing

Unread post by Alexander Halser »

True, the graphics can make a difference as well.
Alexander Halser
Senior Software Architect, EC Software GmbH
Marvin Mueller
Posts: 10
Joined: Tue May 31, 2016 9:29 am

Re: Speed of CHM Publishing

Unread post by Marvin Mueller »

Hi Tim & Alexander,

I replaced all resized graphics with full size ones with regex. Approximately 300 topics were affected by this change. As some topics contained more than one resized image this equals around 500-600 images.

Last night I ran a new (command line) publish-job on the SSD with image-resizing removed. Unfortunately it still took 3:48hrs to complete.

Regards
Marvin
User avatar
Tim Green
Site Admin
Posts: 23153
Joined: Mon Jun 24, 2002 9:11 am
Location: Bruehl, Germany
Contact:

Re: Speed of CHM Publishing

Unread post by Tim Green »

Hi Marvin,
Last night I ran a new (command line) publish-job on the SSD with image-resizing removed. Unfortunately it still took 3:48hrs to complete.
This is bizarre. I think we're going to have to have a look at the project. Unfortunately, since it's a timing issue we're going to need everything, i.e. also the graphics and any snippet files and child projects if you are using those. My guess is that it will be too large to mail even once you've zipped it up. So the best solution would be to put it on Dropbox or another cloud service or your website and mail us the link so that we can download the zip. Mail it to support AT ec-software.com (replace the AT with @)
Regards,
Tim (EC Software Documentation & User Support)

Private support:
Please do not email or PM me with private support requests -- post to the forum directly.
Marvin Mueller
Posts: 10
Joined: Tue May 31, 2016 9:29 am

Re: Speed of CHM Publishing

Unread post by Marvin Mueller »

Hi Tim,

there is some news on the export: Today I hade the chance to publish the help project on a brand new computer. It has an i5-6400 with a SSD, running Windows 8.1. The export took about 2:37hrs.

What I guess from my recent observations is, that the time, a topic needs for publishing, is directly dependent on the number of internal links inside the topic. There are topics with only a few or no links inside, but there are topics with several hundert links inside as well. It seems, that handling of links is heavily dependent on CPU capabilities. This would explain why exporting on a newer i5 is that much faster than on an older i3 processor.

Thank you for the offer to have a look into our projekt. I will talk to my boss about sending you a zipped copy of the project on monday.

Thanks & have a nice weekend
Marvin
User avatar
Alexander Halser
EC-Software Support
Posts: 4098
Joined: Mon Jun 24, 2002 7:24 pm
Location: Salzburg, Austria
Contact:

Re: Speed of CHM Publishing

Unread post by Alexander Halser »

Hello Marvin,

I got your project for testing, thank you! With Help+Manual 6, I can confirm that publishing a CHM file takes about 2 hours. The very same project compiled with Help+Manual 7, however, takes less than 10 minutes!

We have optimized the publishing process in version 7 and were aware that v7 is significantly faster.
This result, however, has surprised me, too :P
Alexander Halser
Senior Software Architect, EC Software GmbH
Post Reply