ASP.NET webservices use XML 1.0 which restricts the character set allowed to the following chars:
 Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF] /* any Unicode character, excluding the surrogate blocks, FFFE, and FFFF. */
As you can see several characters below 0x20 are not allowed with XML 1.0. This also includes characters like the vertical tab (0x0B) which is used pretty frequently.
For backward compatibility reasons .NET webservices do not support XML 1.1 which would allow these character as explained in the following article:
W3C Recommendations NOT Supported at This Time
XML 1.1 – Microsoft has deliberately chosen not to support the XML 1.1 Recommendation unless there is significant customer demand.
Usually the limitation of XML 1.0 is not hurting – except if the XML response sent back to the client would include one of the forbidden characters like the vertical tab.
An interesting tidbit is that the web service stub (server) routines implemented in .NET framework do not bother about the invalid characters when encoding the XML response. They encode the invalid characters as numeric character reference like  for the vertical tab char. The problem occurs in the web service proxy (client) routines. These raise an exception when an entity is returned which is not allowed in XML 1.0:
There is an error in XML document (8, 1314).
at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle)
at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object parameters)
So there is a slight discrepancy between the handling of the invalid characters on the web service proxy and stub side.
It would be great if the ASP.NET webservice classes would avoid sending invalid numeric character references to the client at all – but that is not implemented in .NET framework. As the client also cannot get hands on the content before the exception is raised it would be required to fix the issue in the application logic of the web service itself.
Means each web service method would have to replace the invalid characters before sending the XML content to the client.
The problem here is that usually the automatic XML serialization of managed objects is used to create the XML response. So fixing the issue inside the web service is also not trivial.
This issue also affects the standard sharepoint web services which allow access to SharePoint content.
To overcome this problem it would be required to remove the invalid characters from the XML response (e.g. replace them with a space char) after they have been serialized to XML and before they are sent over the wire to the caller of the web service.
The good message is: ASP.NET has indeed a way to achieve this: SoapExtensions. A SoapExtension allows to consume and modify the SOAP message sent from client to server and vice versa.
Below is a SoapExtension which replaces the invalid control characters with a blank character (0x20) using Regular Expressions:
/// This source code is freeware and is provided on an “as is” basis without warranties of any kind,
/// whether express or implied, including without limitation warranties that the code is free of defect,
/// fit for a particular purpose or non-infringing. The entire risk as to the quality and performance of
/// the code is with the end user.
public class XmlCleanupSoapExtension : SoapExtension
private Regex replaceRegEx;
private Stream oldStream;
private Stream newStream;
// to modify the content we redirect the stream to a memory stream to allow
// easy consumption and modifcation
public override Stream ChainStream(Stream stream)
// keep track of the original stream
oldStream = stream;
// create a new memory stream and configure it as the stream object to use as input and output of the webservice
newStream = new MemoryStream();
public override object GetInitializer(LogicalMethodInfo methodInfo, SoapExtensionAttribute attribute)
// the module is intended to look at all methods. Not on methods tagged with a specific attribute
throw new Exception(“The method or operation is not implemented.”);
public override object GetInitializer(Type serviceType)
// create a compiled instance of the Regular Expression for the chars we would like to replace
// add all char points beween 0 and 31 excluding the allowed white spaces (9=TAB, 10=LF, 13=CR)
StringBuilder RegExp = new StringBuilder(“&#(0”);
for (int i = 1; i <= 31; i++)
// ignore allowed white spaces
if (i == 9 || i == 10 || i == 13) continue;
// add other control characters
// add hex representation as well
string strRegExp = RegExp.ToString();
// create regular expression assembly
Regex regEx = new Regex(strRegExp, RegexOptions.Compiled | RegexOptions.IgnoreCase);
// return the compiled RegEx to all further instances of this class
public override void Initialize(object initializer)
// instance initializers retrieves the compiled regular expression
replaceRegEx = initializer as Regex;
public override void ProcessMessage(SoapMessage message)
if (message.Stage == SoapMessageStage.AfterSerialize)
// process the response sent back to the client – means ensure it is XML 1.0 compliant
if (message.Stage == SoapMessageStage.BeforeDeserialize)
// just copy the XML Soap message from the incoming stream to the outgoing
public void ProcessInput(SoapMessage message)
// no manipulation required on input data
// copy content from http stream to memory stream to make it available to the web service
TextReader reader = new StreamReader(oldStream);
TextWriter writer = new StreamWriter(newStream);
// set position back to the beginning to ensure that the web service reads the content we just copied
newStream.Position = 0;
public void ProcessOutput(SoapMessage message)
// rewind stream to ensure that we read from the beginning
newStream.Position = 0;
// copy the content of the stream into a memory buffer
byte buffer = (newStream as MemoryStream).ToArray();
// shortcut if stream is empty to avoid exception later
if (buffer.Length == 0) return;
// convert buffer to string to allow easy string manipulation
string content = Encoding.UTF8.GetString(buffer);
// replace invalid XML entities using regular expression
content = replaceRegEx.Replace(content, “ ”);
// convert back to byte buffer
buffer = Encoding.UTF8.GetBytes(content);
// stream byte buffer to the client app
oldStream.Write(buffer, 0, buffer.Length);
The above code should be compiled into a C# class library project and signed with a strong name to allow placing the DLL into a GAC.
Afterwards the SoapExtension can be registered in the web.config of the affected web service. For SharePoint webservices this would be the web.config in the following directory:
C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\ISAPI\
The following entry needs to be added to the web.config:
<?xml version=”1.0″ encoding=”UTF-8″ standalone=”yes”?>
<add type=“StefanG.SoapExtensions.XmlCleanupSoapExtension, XmlCleanupSoapExtension, Version=184.108.40.206, Culture=neutral, PublicKeyToken=0e15300fe8a7b210” priority=“1” group=“0” />
Afterwards all responses from webservice methods in the affected web application will automatically be cleaned up.
The complete source code can also be downloaded from here: http://code.msdn.microsoft.com/XmlCleanupSoapExtens