When I consult with development teams, at some point performance always rears its ugly head. When I code review something I often find it overly complex with lots of caching, “clever” code etc. and when I ask why the code is this complex the answer is usually for performance, E.g. “we need to cache the results as that’s an expensive operation”.
At this point I usually ask if they have any evidence to show how expensive the operation is, all too often the answer is “no”. Once again another developer is the victim of premature optimisation.
Now don’t get me wrong, optimising code is important, things like caching can indeed help reduce the cost of expensive operations. However, if that operation is only called once is caching really going to help? If the operation is “expensive” but only takes 2ms to execute and is called next to an operation that takes 2s is optimising it really going to help? I hope you all answered no there!
So how do we find out if we need to optimise, where do we focus our effort with limited time and resources to get the best return? That’s where profilers come in. If you’ve not used one before, they are applications that monitor how an application runs, usually capturing data such as number of calls, time taken, memory allocated etc.
Visual Studio comes with some pretty good profiling tools in the Team System version that can really help. There is a nice Performance Wizard on the Analyze menu that does much of the work for you. But as nice as the wizard is, it doesn’t fit all situations. If you need to profile services, code running as different users, on servers without VS installed or just need more control then the command line tools is where it’s at. You can download a standalone version from http://www.microsoft.com/downloads/details.aspx?familyid=fd02c7d6-5306-41f2-a1be-b7dcb74c9c0b&displaylang=en for installation on servers or machines without VS.
In this post I’m going to go through the (simple) steps needed to profile an application, I’ll leave interpreting the reports to another post.
Types of profiling
There are two types of profiling you can do, sampling and instrumentation. I’m going to mainly cover instrumentation as that’s the one I usually use as it’s the most accurate, it is however also the most invasive and requires changing assemblies so isn’t always suitable.
Setting up the profiler
If you have Visual Studio Team System for Developers then you have the profiler installed. The first thing to do is configure the command prompt to let you use the profiling tools. You’ll need to add them into your path. They are located by default in “C:\Program Files\Microsoft Visual Studio 9.0\Team Tools\Performance Tools” but it may be a different location on 64bit (Program Files (x86)) or if you' installed to a none standard location.
Once that’s done you need to enable the environment variables the profiler needs. You can do this with the VSPerfCLREnv command. You want to use /traceon if you are able to trace the application from the command console window by launching it but if you are tracing a service then use /globaltraceon. The global one will most likely need a reboot for the service to pick up the new settings so I usually do that afterwards.
Note if you are using sampling then you use /sampleon and /globalsampleon instead.
Preparing the assemblies
Now that the profiler is configured you need to prep the assemblies as we are instrumenting and not sampling. If you are sampling you can miss this step out.
Build your assemblies as normal, ensure you have pdb files generated though as you’ll need those later to view the source code from the profiling reports.
Now instrument all of the assemblies you want performance data from, you use VSInstr to do this. Simply specify the assembly name and you’ll get a new assembly and pdb file, the original ones get renamed to .orig.
Note that if you do this to a strong named assembly it’ll break the signing, you’ll need to use sn with the –Vr option to skip strong name verification on the assemblies you have instrumented. Don’t forgot to switch it back once you are done.
Profiling
Now you’re all ready to profile. To control the profiler you simply start it using the VSPerfCmd command. You specify the /start:trace option to start instrumenting or /start:sample to start sampling. You will also need /output to set the output filename for the trace results.
E.g. VSPerfCmd /start:trace /output:traceoutput.vsp
If you are tracing a service you’ll also need to add /user: and probably /crosssession as the service will be running as a different user and in a different session from the profiler. You’ll need to have the permissions to do this.
Once the profiler is started simply now run up your application and use it. It's important to make sure that the application is not running until after the profiler is started, this is often missed when profiling IIS, stop the app pool or do an IISReset before starting the profiler.
When you’ve done you need to shutdown the profiler, simply issue VSPerfCmd /shutdown to do that.
You should now have a shiny new .vsp file that contains the trace data. You can open this in Visual Studio, however the chances are you’ll just see stream of hex addresses and no source information.
I’m packing symbols
To make a .vsp file that can link back to source you need symbols, remember I told you that you’ll need these so if you don’t have them go back and start again and tell yourself that you’re a very naughty boy/girl (delete as appropriate).
How do we get the symbols in there or what if I don’t have Visual Studio? You can use the VSPerfReport command to pack the symbols into the .vsp file and to generate a text report of the trace.
You want to ensure you have your symbols set up to pull down from a symbol server to get all of them for the core .NET bits. To do that set the environment variable _NT_SYMBOL_PATH to point to the symbol server and your local cache.
E.g. set _NT_SYMBOL_PATH=symsrv*symsrv.dll*c:\mylocalcache*http://msdl.microsoft.com/download/symbols
See http://support.microsoft.com/kb/311503 for more details on setting the symbol path.
I usually just do a VSPerfReport /packsymbols /summary:function /symbolpath=<path to .instr.pdb files> traceoutput.vsp to get my symbols packed. This will also generate a summary text file, you need to do that even if you don’t need the summary or the symbols will not get packed. If you want a more detailed report then you can use the /summary option to specify what information you want in CSV format.
Comparing runs
Another nice feature of VSPerfReport is the /diff option. This lets you compare two .vsp files and shows the differences, this can be useful for checking against a baselined performance session to see if you’ve made things better or worse.
And finally…
Don’t forget that you want to turn instrumentation off if you are running it on a production server, replace the assemblies with the uninstrumented versions, revert the sn commands if you used them, run VSPerfCLREnv /off to clean up environement variables.
I’ll try and post something about interpreting the reports that the profiler gives you soon, after all doing all of this doesn’t help if you can’t read the data!