I found a nasty bug in a BizTalk pipeline component that I put together and has been happily running in live for a while. I was getting out of memory exceptions, which was a surprise as the messages are fairly small, and I was wrapping the incoming message in a ReadOnlySeekableStream over a VirtualStream.
The component is a debatcher, and it lets you specify a map to execute on all the debatched items, so by the time they hit the message box they've already been transformed. The code for that is straightforward:
var inputDocument = new XmlDocument();
inputDocument.LoadXml(element);
using (MemoryStream outputStream = new MemoryStream())
{
_Transform.Transform.Transform(inputDocument, _Transform.TransformArgs, outputStream);
outputStream.Flush();
outputStream.Position = 0;
using (StreamReader reader = new StreamReader(outputStream))
{
output = new StreamReader(outputStream).ReadToEnd();
}
}
And to save me loading the map from the type name every time I need it, I keep it in a static variable:
private static TransformBase _Transform;
private static object _TransformLock = new object();
//…
lock (_TransformLock)
{
var transformType = Type.GetType(RecordTransormFullyQualifiedTypeName);
if (transformType != null)
{
_Transform = Activator.CreateInstance(transformType) as TransformBase;
}
}
Which was all fine until we ramped up the load. When the incoming message had 5,000 records totalling about 200Kb, which is pretty tiny, I was getting OutOfMemory exceptions at the line:
_Transform.Transform.Transform(inputDocument, _Transform.TransformArgs, outputStream);
I’d assumed that the XLANGs TransformBase class doesn’t do anything complicated when you call .Transform to get to the underlying XslTransfom, but actually there’s a pretty involved stack which loads the XSL schema then does compilation and serialization, and those steps don’t clear out the memory as my TransformBase is a static:
at System.Xml.Xsl.XsltOld.Compiler.CompileAssembly(ScriptingLanguage lang, Hashtable typeDecls, String nsName, Evidence evidence)
at System.Xml.Xsl.XsltOld.Compiler.CompileScript(Evidence evidence)
at System.Xml.Xsl.XsltOld.Compiler.Compile(NavigatorInput input, XmlResolver xmlResolver, Evidence evidence)
at System.Xml.Xsl.XslTransform.Compile(XPathNavigator stylesheet, XmlResolver resolver, Evidence evidence)
at System.Xml.Xsl.XslTransform.Load(XPathNavigator stylesheet, XmlResolver resolver, Evidence evidence)
at System.Xml.Xsl.XslTransform.Load(XmlReader stylesheet, XmlResolver resolver, Evidence evidence)
at Microsoft.XLANGs.BaseTypes.TransformBase.get_Transform()
So with a 200Kb file the BTSNTSvc exe ramped up to 1.5Gb memory usage and started throwing the OOM exceptions. The process didn’t crash, but BizTalk didn’t handle it too well so I had to kill the process.
The fix is simple – cache the XslTransform instance in the static variable rather than the TransformBase, so the code now looks like this:
private static XslTransform _Transform;
private static XsltArgumentList _TransformArgs;
private static object _TransformLock = new object();
//…
lock (_TransformLock)
{
Type transformType = Type.GetType(TransormFullyQualifiedTypeName);
if (transformType != null)
{
TransformBase transform = Activator.CreateInstance(transformType) as TransformBase;
_Transform = transform.Transform;
_TransformArgs = transform.TransformArgs;
}
}
//…
XmlDocument inputDocument = new XmlDocument();
inputDocument.LoadXml(element);
using (MemoryStream outputStream = new MemoryStream())
{
_Transform.Transform(inputDocument, _TransformArgs, outputStream);
outputStream.Flush();
outputStream.Position = 0;
using (StreamReader reader = new StreamReader(outputStream))
{
output = new StreamReader(outputStream).ReadToEnd();
}
}
We only get the hit of compiling the
XslTransform on the first use of the map, and it scales nicely with much larger files.