AppTimer by Passmark : bench-marking application start-up-times in Windows

AppTimer is a small, free utility which can be used to measure the start-up-time of applications.  It can launch a program and measure the time it took, to be ready to accept user input. It could also close the program automatically and repeat the test a specified number of times.

As you can see in the above picture you have the option of supplying command line arguments to the program being run. You must specify a log file, the log file is where AppTimer stores the results. The Window name is the name of the main window of the program being run. The window names do not have to match exactly; AppTimer supports partial matching.

In the executions box, you can specify the number of times the program should be run. The delay option is for giving the program being run time to clean up properly between repeat executions. A bigger program will require a longer delay.

In the Window Detection method, I have checked, Input Idle, Window Name and Visible. This means that AppTimer would wait for the window to become visible and ready to receive user input. You could try out other combinations as well if that combination does not work.

Please note that if you have enabled the Visible option or any of the windows closing options, explained below, you must also enable the Window Name detection option as a Window Detection Method. This is a bit confusing but AppTimer would show an error pop-up and refuse to run the program otherwise.

In the Window Close Method, I have chosen WM_CLOSE. The WM_SYSCOMMAND option sends a  WM_SYSCOMMAND message to the window with SC_CLOSE as argument. This did not work for me for some reason. The Alt+F4 option would try to send that key combination to the window. Like in the case of window detection, you have to try the various options to see which ones work for you since its success is dependent on the program being run.

After you have set up everything click Run App. AppTimer would run the program as many times as you have specified and write the results to the log file.

I tried to compare Gvim’s start-up-time without any plugins with Gvim’s start-up-time with only Youcompleteme plugin loaded. Here are the results from their respective log files.

Unfortunately, AppTimer writes only the name of the program in its log file and not the full command which was used to invoke the program. This can make things a little confusing when you are running the same program with different arguments as the case above.

Launching an application multiple times like this to measure its start-up-time may not be a good choice in many cases. In the case of large applications, caching could speed up the repeat runs so much that the results would become completely unrealistic. In the Gvim example above, the difference between the first execution and the subsequent executions is quite small, so I believe the results are valid in this case. For benchmarking large applications you may have to set the number of executions to 1 and repeat the tests manually with long intervals between them. AppTimer would append the result of such repeated tests to the same log file. Another option would be to use AppTimer’s config file. AppTimer does support running a benchmark using a config file and the config file has an autorun flag. You can combine this option with some scripting to automate the process. Not a very convenient option but something worth considering I think.

Using a config file

The parameters for running an AppTimer benchmark can be supplied using a config file. The config file must be named config.txt and must be in the same directory as the AppTimer executable. In the config file, each parameter must be specified in a separate line and it must be specified in a particular order. Fortunately, the order in which the parameters must be specified roughly corresponds to their order in the GUI. If a parameter has no value, then you must leave the line blank. For options, use 1 to select an option and 0 to de-select.

The config file parameters which I wrote for the Gvim test and a description of the parameters is given below.