Control Characters in XML and Soap

Problem:

I have a character string that is data received from a third party which contains control characters (specifically Ox19) within it.

The character can appear at any point and I cannot replace it with another character without invalidating the data completely.

XML 1.0 considers control characters as invalid and throws an error on reaching the character.

Can anyone suggest ANY sort of fix for this problem? Even some sort of nasty botch would be acceptable!!!

Thanks
Mitchell x

Well, I suppose it depends on what you're doing with the string. Can you wrap it in <![CDATA[ ]]> tags?

Thanks for the reply

Great idea, tried it...., didn't work

As far as what I'm doing with the data is concerned...

I am generating a soap/xml document by serialising an object, this actually works fine and doesn't complain about the invalid Ox19 character. When I try to deserialise the soap/xml back into the object it fails while simply trying to parse the file because XML version 1.0 doesn't allow control characters! Flippin thing!

I'm going to try posting on the C# forum as well to see if there is a way round this problem that way but any further suggestions would be most welcome

Thanks again
mitchell x

Well, I'm at a loss. One thing you might try is to pass values instead of the whole serialized object. It's harder, but since serializing doesn't seem to be working...

Hi mitchell,

I was running into this exact problem. When .NET sends soap requests, the xml isn't validated. The back-end of our project handles these invalid characters without a problem, and subsequent response messages included these invalid characters which makes .NET barf.

The only consistent way that I found to work around this problem is to write a soapextension that intercepts each message before it's deserialized. There's lots of articles out there that tell you how to create a soapextension, so I won't go into how soapextensions work here... but the guts of it is in the ProcessMessage method.

public override void ProcessMessage(SoapMessage message) {
MemoryStream reallyNewStream;
switch (message.Stage)
{
case SoapMessageStage.BeforeSerialize:
break;
case SoapMessageStage.AfterSerialize:
newStream.Position = 0;
// copy the new stream (which has the serialized message) into the oldstream so that the thing will be processed
CopyStream(newStream, oldStream);
break;
case SoapMessageStage.BeforeDeserialize:
// copy the old stream (which has the serialized message) into the newstream so that we can parse it...
CopyStream(oldStream, newStream);
reallyNewStream = new MemoryStream();
// replace all invalid chars in the soap response
ReplaceInvalidChars(newStream, reallyNewStream);
// copy the new, valid soap response into the newStream object so it can continue on it's merry way
CopyStream(reallyNewStream, newStream);
newStream.Position=0;
break;
case SoapMessageStage.AfterDeserialize:
break;
default:
throw new Exception("invalid stage");
}
}

The main problem with this is the fact that you have to go through each character on each and every soap request. Not pretty, but it works. I've been posting on different forums, and I haven't found a more elegant way of doing it.

Hope this helps.

Mike

Another idea occured to me. Base64 or uuencode the serialized object before transmission. Then decode it on the other side. It means more work on each end of the transmission but it should handle all exception cases and be fairly portable.