Any suggestions for optimising speed of content compiling?

Please post all questions and comments regarding Help & Manual 7 here.

Moderators: Alexander Halser, Tim Green

Post Reply
User avatar
itexnz
Posts: 137
Joined: Tue Jan 06, 2009 10:09 pm
Location: Wellington, New Zealand
Contact:

Any suggestions for optimising speed of content compiling?

Unread post by itexnz »

Hi, I have a decent computer with a metric crap-ton of RAM and trying to determine the best way to get my outputs to regenerate super quickly...

e.g. during my scripts to rebuild all my help and PDF files I never see the CPUs rise about 20% for HELPMAN.EXE and never see the RAM utilisation or IO for that process or related processes show any bottleneck

I also utilise a RAM drive for system and user \temp\ files (as my computer has so much RAM its silly not to)

Is there anything else I can do to optimise the processes to go faster? my scripts for about a dozen PDF publishing destinations and maybe 1000 web pages takes about 15 minutes... and well, you know what developers are like, they want their updated content committed like five minutes ago (unfortunately they do not use a build server or publish help to the web for such things. Sigh.)
David Scott
Documentation Infrastructure Consultant
https://www.sourceone.co.nz
SourceOne. Documentation, engineered.
User avatar
Tim Green
Site Admin
Posts: 23156
Joined: Mon Jun 24, 2002 9:11 am
Location: Bruehl, Germany
Contact:

Re: Any suggestions for optimising speed of content compilin

Unread post by Tim Green »

Hi David,

Fifteen minutes for that load sounds pretty much par for the course. What takes time is not the processing -- as you see yourself the processor load is not all that high -- but all the physical IO of reading and writing all the files. In PDF in particular very large temporary files need to be written while rendering the final PDF, and then translated to PDF, and that takes a while. PDF rendering time will double if you use Page X of Y in your page numbering scheme, because then the entire document needs to be rendered twice to get the page references right.

What will slow your process down most is doing anything at all on a network drive. That is always a bottleneck and it will increase rendering time by orders of magnitude. The next factor is drive speed: Ideally everything should be on the fastest local SSD drives possible, and they should all be installed inside your local computer. External USB drives will also be much slower, including USB3, even if they are SSDs. External USB-C-Thunderbolt SSD drives can sometimes be as fast as internal SSD, but only if everything is working correctly, and on Windows PCs it often isn't. External USB-C/Thunderbolt connections still very wildly in their performance, so it's not possible to say definitively that they are faster or even fast enough, at least not yet.

Of course, the computer should also be fast enough to make full use of the I/O capabilities of really fast SSDs, but if you're investing in drives like that, it's unlikely that the computer will be far behind.

We've done testing with RAM drives and they only provide very marginal improvements over current SSDs. On balance, they're not really worth the trouble.
Regards,
Tim (EC Software Documentation & User Support)

Private support:
Please do not email or PM me with private support requests -- post to the forum directly.
User avatar
itexnz
Posts: 137
Joined: Tue Jan 06, 2009 10:09 pm
Location: Wellington, New Zealand
Contact:

Re: Any suggestions for optimising speed of content compilin

Unread post by itexnz »

Thanks, good to know that at least I'm not doing something wrong or missed an obvious optimisation opportunity :)
David Scott
Documentation Infrastructure Consultant
https://www.sourceone.co.nz
SourceOne. Documentation, engineered.
Post Reply