Parsing XML from TCP connectionParsing concatenated, non-delimited XML messages from TCP-stream using C#How do I parse XML from NetworkStream via ReadAsync of XmlReader in C#?How does one parse XML files?Get int value from enum in C#What characters do I need to escape in XML documents?How do I parse XML in Python?What does <![CDATA[]]> in XML mean?Parsing concatenated, non-delimited XML messages from TCP-stream using C#How do you parse and process HTML/XML in PHP?Sending large objects over TCP: “End of Stream encountered before parsing was completed”Sending and Receiving XML data over TCPParsing XML string coming from a TCP/IP connection

Identification of vintage sloping window

How to take the beginning and end parts of a list with simpler syntax?

Why is transplanting a specific intact brain impossible if it is generally possible?

What is the length of pair of wires after twisting them around each other?

On the Rømer experiments and the speed if light

constant evaluation when using differential equations.

Are differences between uniformly distributed numbers uniformly distributed?

How can I solve for the intersection points of two ellipses?

Is it legal for a company to enter an agreement not to hire employees from another company?

As a 16 year old, how can I keep my money safe from my mother?

Who are these characters/superheroes in the posters from Chris's room in Family Guy?

How is this kind of structure made?

Continuous vertical line using booktabs in tabularx table?

Why did the RAAF procure the F/A-18 despite being purpose-built for carriers?

try/finally with bash shell

How should an administrative assistant reply to student addressing them as "Professor" or "Doctor"?

Why isn’t SHA-3 in wider use?

How can Radagast come across Gandalf and Thorin's company?

Tikzpicture - finish drawing a curved line for a cake slice

A simple stop watch which I want to extend

Bitcoin successfully deducted on sender wallet but did not reach receiver wallet

How are you supposed to know the strumming pattern for a song from the "chord sheet music"?

Can you castle with a "ghost" rook?

What are the conventions for transcribing Semitic languages into Greek?



Parsing XML from TCP connection


Parsing concatenated, non-delimited XML messages from TCP-stream using C#How do I parse XML from NetworkStream via ReadAsync of XmlReader in C#?How does one parse XML files?Get int value from enum in C#What characters do I need to escape in XML documents?How do I parse XML in Python?What does <![CDATA[]]> in XML mean?Parsing concatenated, non-delimited XML messages from TCP-stream using C#How do you parse and process HTML/XML in PHP?Sending large objects over TCP: “End of Stream encountered before parsing was completed”Sending and Receiving XML data over TCPParsing XML string coming from a TCP/IP connection






.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;








0















I've run into issue similar to this - I'm trying to parse XML received from TCP without any delimiters and any way of knowing did I receive complete or partial XML. For now, I've taken following approach - keep adding received data to string, try to split it with var xmls = Regex.Split(newData, @"(?=<?xml)"); and try to parse each of strings that came from split - if parsed correctly, it's ok but if not, put the non-complete XML to input buffer and continue. Here is part of receive code that does that:



 string inputBuffer = "";
private void TcpEndReceive(IAsyncResult result)

lock (tcpLock)

int bytesAvailable = 0;
try

bytesAvailable = tcpClient.GetStream().EndRead(result);

catch (Exception ex)

log.Error("TcpEndReceive: error endRead, ", ex);
OnStatusChanged(EReceiverStatus.ConnectionError);
TcpClose();


if (bytesAvailable >= 0)

try

var newData = System.Text.ASCIIEncoding.ASCII.GetString(tcp_data, 0, bytesAvailable);

if (!ParseMessage(newData))

var xmls = Regex.Split(newData, @"(?=<?xml)");
foreach (var xml in xmls)

if (xml.Length == 0) continue;
if (!ParseMessage(xml))

inputBuffer += xml;



if (inputBuffer.Length == 0) return;
if (ParseMessage(inputBuffer))

inputBuffer = "";



catch (Exception ex)

log.Error("TcpEndReceive:error reading data, ", ex);
OnStatusChanged(EReceiverStatus.ConnectionError);
TcpClose();



TcpBeginReceive();



Now, this may look fine but it isn't - if part of message gets out-of-order, this won't work. Anyone has any idea how to solve this? ParseMessage returns true if message is parsed ok. Also, it can happen that you receive part of message, whole message (yay!) or even multiple messages (XMLs) as single transmit.



Any suggestions, ideas or help is welcome. Request is "simple" - receive full or partial XMLs from TCP stream, parse ones that work and wait for other pieces to arrive for those that can't be parsed. Only reliable "delimiter" you can use is <?xml at beginning of every (valid) XML.



[edit] - Additional clarification - I have no control over data that is sent to me - I can't add separators, delimiters or anything. I can only receive strings that may or may be not complete XML. Some messages contain Base64 encoded images and they don't fit in one message - first you get start of XML, then few ms later you get Base64 image (without any indication that more data follows) and then you get end of your XML. As I said, only reliable signal is <?xml at start of valid XML (and beginning of next one, of course). Also, there is a slight problem that there is multiple types of message, with different root elements, so you can't just deserialize to class Message - you have 10-15 different ones. ParseMessage in my example takes the string, finds root element and first node and depending on it deserializes the message in something useful.










share|improve this question


























  • Do not parse xml until the entire TCP message is received. When sending TCP you need to do one of following to make sure receiver know where the end of message occurs 1) ASCII : Terminate message with a character not part of the data 2) ASCII or Binary : Add a byte count to beginning of the message 3) Send Fixed length message

    – jdweng
    Mar 27 at 9:00











  • @jdweng from the question, though, it sounds like the source isn't up to the OP ("I'm trying to parse XML received from TCP without any delimiters "). Personally, though, I'd agree - the best "fix" here is to change the data source to not be so pathologically horrible

    – Marc Gravell
    Mar 27 at 9:01











  • The OP says : "keep adding received data to string". The xml identification line only occurs once so you can't split on the identification. You have to wait for entire xml file to be received. So you should not be doing the split until entire file is received. The root cause of issue is to make sure you receive the entire message before doing any processing (except joining the received strings).

    – jdweng
    Mar 27 at 9:16











  • Maybe I wasn't clear enough (I'll add to post) - I have no control over received data and there are no any kind of delimiters - messages just keep arriving in packets, sometimes whole message (short ACK) or truncated (containing Base64 encoded images) so you need to wait multiple TCP receives to get whole message.

    – nighthawk
    Mar 27 at 14:20

















0















I've run into issue similar to this - I'm trying to parse XML received from TCP without any delimiters and any way of knowing did I receive complete or partial XML. For now, I've taken following approach - keep adding received data to string, try to split it with var xmls = Regex.Split(newData, @"(?=<?xml)"); and try to parse each of strings that came from split - if parsed correctly, it's ok but if not, put the non-complete XML to input buffer and continue. Here is part of receive code that does that:



 string inputBuffer = "";
private void TcpEndReceive(IAsyncResult result)

lock (tcpLock)

int bytesAvailable = 0;
try

bytesAvailable = tcpClient.GetStream().EndRead(result);

catch (Exception ex)

log.Error("TcpEndReceive: error endRead, ", ex);
OnStatusChanged(EReceiverStatus.ConnectionError);
TcpClose();


if (bytesAvailable >= 0)

try

var newData = System.Text.ASCIIEncoding.ASCII.GetString(tcp_data, 0, bytesAvailable);

if (!ParseMessage(newData))

var xmls = Regex.Split(newData, @"(?=<?xml)");
foreach (var xml in xmls)

if (xml.Length == 0) continue;
if (!ParseMessage(xml))

inputBuffer += xml;



if (inputBuffer.Length == 0) return;
if (ParseMessage(inputBuffer))

inputBuffer = "";



catch (Exception ex)

log.Error("TcpEndReceive:error reading data, ", ex);
OnStatusChanged(EReceiverStatus.ConnectionError);
TcpClose();



TcpBeginReceive();



Now, this may look fine but it isn't - if part of message gets out-of-order, this won't work. Anyone has any idea how to solve this? ParseMessage returns true if message is parsed ok. Also, it can happen that you receive part of message, whole message (yay!) or even multiple messages (XMLs) as single transmit.



Any suggestions, ideas or help is welcome. Request is "simple" - receive full or partial XMLs from TCP stream, parse ones that work and wait for other pieces to arrive for those that can't be parsed. Only reliable "delimiter" you can use is <?xml at beginning of every (valid) XML.



[edit] - Additional clarification - I have no control over data that is sent to me - I can't add separators, delimiters or anything. I can only receive strings that may or may be not complete XML. Some messages contain Base64 encoded images and they don't fit in one message - first you get start of XML, then few ms later you get Base64 image (without any indication that more data follows) and then you get end of your XML. As I said, only reliable signal is <?xml at start of valid XML (and beginning of next one, of course). Also, there is a slight problem that there is multiple types of message, with different root elements, so you can't just deserialize to class Message - you have 10-15 different ones. ParseMessage in my example takes the string, finds root element and first node and depending on it deserializes the message in something useful.










share|improve this question


























  • Do not parse xml until the entire TCP message is received. When sending TCP you need to do one of following to make sure receiver know where the end of message occurs 1) ASCII : Terminate message with a character not part of the data 2) ASCII or Binary : Add a byte count to beginning of the message 3) Send Fixed length message

    – jdweng
    Mar 27 at 9:00











  • @jdweng from the question, though, it sounds like the source isn't up to the OP ("I'm trying to parse XML received from TCP without any delimiters "). Personally, though, I'd agree - the best "fix" here is to change the data source to not be so pathologically horrible

    – Marc Gravell
    Mar 27 at 9:01











  • The OP says : "keep adding received data to string". The xml identification line only occurs once so you can't split on the identification. You have to wait for entire xml file to be received. So you should not be doing the split until entire file is received. The root cause of issue is to make sure you receive the entire message before doing any processing (except joining the received strings).

    – jdweng
    Mar 27 at 9:16











  • Maybe I wasn't clear enough (I'll add to post) - I have no control over received data and there are no any kind of delimiters - messages just keep arriving in packets, sometimes whole message (short ACK) or truncated (containing Base64 encoded images) so you need to wait multiple TCP receives to get whole message.

    – nighthawk
    Mar 27 at 14:20













0












0








0








I've run into issue similar to this - I'm trying to parse XML received from TCP without any delimiters and any way of knowing did I receive complete or partial XML. For now, I've taken following approach - keep adding received data to string, try to split it with var xmls = Regex.Split(newData, @"(?=<?xml)"); and try to parse each of strings that came from split - if parsed correctly, it's ok but if not, put the non-complete XML to input buffer and continue. Here is part of receive code that does that:



 string inputBuffer = "";
private void TcpEndReceive(IAsyncResult result)

lock (tcpLock)

int bytesAvailable = 0;
try

bytesAvailable = tcpClient.GetStream().EndRead(result);

catch (Exception ex)

log.Error("TcpEndReceive: error endRead, ", ex);
OnStatusChanged(EReceiverStatus.ConnectionError);
TcpClose();


if (bytesAvailable >= 0)

try

var newData = System.Text.ASCIIEncoding.ASCII.GetString(tcp_data, 0, bytesAvailable);

if (!ParseMessage(newData))

var xmls = Regex.Split(newData, @"(?=<?xml)");
foreach (var xml in xmls)

if (xml.Length == 0) continue;
if (!ParseMessage(xml))

inputBuffer += xml;



if (inputBuffer.Length == 0) return;
if (ParseMessage(inputBuffer))

inputBuffer = "";



catch (Exception ex)

log.Error("TcpEndReceive:error reading data, ", ex);
OnStatusChanged(EReceiverStatus.ConnectionError);
TcpClose();



TcpBeginReceive();



Now, this may look fine but it isn't - if part of message gets out-of-order, this won't work. Anyone has any idea how to solve this? ParseMessage returns true if message is parsed ok. Also, it can happen that you receive part of message, whole message (yay!) or even multiple messages (XMLs) as single transmit.



Any suggestions, ideas or help is welcome. Request is "simple" - receive full or partial XMLs from TCP stream, parse ones that work and wait for other pieces to arrive for those that can't be parsed. Only reliable "delimiter" you can use is <?xml at beginning of every (valid) XML.



[edit] - Additional clarification - I have no control over data that is sent to me - I can't add separators, delimiters or anything. I can only receive strings that may or may be not complete XML. Some messages contain Base64 encoded images and they don't fit in one message - first you get start of XML, then few ms later you get Base64 image (without any indication that more data follows) and then you get end of your XML. As I said, only reliable signal is <?xml at start of valid XML (and beginning of next one, of course). Also, there is a slight problem that there is multiple types of message, with different root elements, so you can't just deserialize to class Message - you have 10-15 different ones. ParseMessage in my example takes the string, finds root element and first node and depending on it deserializes the message in something useful.










share|improve this question
















I've run into issue similar to this - I'm trying to parse XML received from TCP without any delimiters and any way of knowing did I receive complete or partial XML. For now, I've taken following approach - keep adding received data to string, try to split it with var xmls = Regex.Split(newData, @"(?=<?xml)"); and try to parse each of strings that came from split - if parsed correctly, it's ok but if not, put the non-complete XML to input buffer and continue. Here is part of receive code that does that:



 string inputBuffer = "";
private void TcpEndReceive(IAsyncResult result)

lock (tcpLock)

int bytesAvailable = 0;
try

bytesAvailable = tcpClient.GetStream().EndRead(result);

catch (Exception ex)

log.Error("TcpEndReceive: error endRead, ", ex);
OnStatusChanged(EReceiverStatus.ConnectionError);
TcpClose();


if (bytesAvailable >= 0)

try

var newData = System.Text.ASCIIEncoding.ASCII.GetString(tcp_data, 0, bytesAvailable);

if (!ParseMessage(newData))

var xmls = Regex.Split(newData, @"(?=<?xml)");
foreach (var xml in xmls)

if (xml.Length == 0) continue;
if (!ParseMessage(xml))

inputBuffer += xml;



if (inputBuffer.Length == 0) return;
if (ParseMessage(inputBuffer))

inputBuffer = "";



catch (Exception ex)

log.Error("TcpEndReceive:error reading data, ", ex);
OnStatusChanged(EReceiverStatus.ConnectionError);
TcpClose();



TcpBeginReceive();



Now, this may look fine but it isn't - if part of message gets out-of-order, this won't work. Anyone has any idea how to solve this? ParseMessage returns true if message is parsed ok. Also, it can happen that you receive part of message, whole message (yay!) or even multiple messages (XMLs) as single transmit.



Any suggestions, ideas or help is welcome. Request is "simple" - receive full or partial XMLs from TCP stream, parse ones that work and wait for other pieces to arrive for those that can't be parsed. Only reliable "delimiter" you can use is <?xml at beginning of every (valid) XML.



[edit] - Additional clarification - I have no control over data that is sent to me - I can't add separators, delimiters or anything. I can only receive strings that may or may be not complete XML. Some messages contain Base64 encoded images and they don't fit in one message - first you get start of XML, then few ms later you get Base64 image (without any indication that more data follows) and then you get end of your XML. As I said, only reliable signal is <?xml at start of valid XML (and beginning of next one, of course). Also, there is a slight problem that there is multiple types of message, with different root elements, so you can't just deserialize to class Message - you have 10-15 different ones. ParseMessage in my example takes the string, finds root element and first node and depending on it deserializes the message in something useful.







c# xml tcpclient






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Mar 27 at 14:31







nighthawk

















asked Mar 27 at 8:33









nighthawknighthawk

1161 silver badge15 bronze badges




1161 silver badge15 bronze badges















  • Do not parse xml until the entire TCP message is received. When sending TCP you need to do one of following to make sure receiver know where the end of message occurs 1) ASCII : Terminate message with a character not part of the data 2) ASCII or Binary : Add a byte count to beginning of the message 3) Send Fixed length message

    – jdweng
    Mar 27 at 9:00











  • @jdweng from the question, though, it sounds like the source isn't up to the OP ("I'm trying to parse XML received from TCP without any delimiters "). Personally, though, I'd agree - the best "fix" here is to change the data source to not be so pathologically horrible

    – Marc Gravell
    Mar 27 at 9:01











  • The OP says : "keep adding received data to string". The xml identification line only occurs once so you can't split on the identification. You have to wait for entire xml file to be received. So you should not be doing the split until entire file is received. The root cause of issue is to make sure you receive the entire message before doing any processing (except joining the received strings).

    – jdweng
    Mar 27 at 9:16











  • Maybe I wasn't clear enough (I'll add to post) - I have no control over received data and there are no any kind of delimiters - messages just keep arriving in packets, sometimes whole message (short ACK) or truncated (containing Base64 encoded images) so you need to wait multiple TCP receives to get whole message.

    – nighthawk
    Mar 27 at 14:20

















  • Do not parse xml until the entire TCP message is received. When sending TCP you need to do one of following to make sure receiver know where the end of message occurs 1) ASCII : Terminate message with a character not part of the data 2) ASCII or Binary : Add a byte count to beginning of the message 3) Send Fixed length message

    – jdweng
    Mar 27 at 9:00











  • @jdweng from the question, though, it sounds like the source isn't up to the OP ("I'm trying to parse XML received from TCP without any delimiters "). Personally, though, I'd agree - the best "fix" here is to change the data source to not be so pathologically horrible

    – Marc Gravell
    Mar 27 at 9:01











  • The OP says : "keep adding received data to string". The xml identification line only occurs once so you can't split on the identification. You have to wait for entire xml file to be received. So you should not be doing the split until entire file is received. The root cause of issue is to make sure you receive the entire message before doing any processing (except joining the received strings).

    – jdweng
    Mar 27 at 9:16











  • Maybe I wasn't clear enough (I'll add to post) - I have no control over received data and there are no any kind of delimiters - messages just keep arriving in packets, sometimes whole message (short ACK) or truncated (containing Base64 encoded images) so you need to wait multiple TCP receives to get whole message.

    – nighthawk
    Mar 27 at 14:20
















Do not parse xml until the entire TCP message is received. When sending TCP you need to do one of following to make sure receiver know where the end of message occurs 1) ASCII : Terminate message with a character not part of the data 2) ASCII or Binary : Add a byte count to beginning of the message 3) Send Fixed length message

– jdweng
Mar 27 at 9:00





Do not parse xml until the entire TCP message is received. When sending TCP you need to do one of following to make sure receiver know where the end of message occurs 1) ASCII : Terminate message with a character not part of the data 2) ASCII or Binary : Add a byte count to beginning of the message 3) Send Fixed length message

– jdweng
Mar 27 at 9:00













@jdweng from the question, though, it sounds like the source isn't up to the OP ("I'm trying to parse XML received from TCP without any delimiters "). Personally, though, I'd agree - the best "fix" here is to change the data source to not be so pathologically horrible

– Marc Gravell
Mar 27 at 9:01





@jdweng from the question, though, it sounds like the source isn't up to the OP ("I'm trying to parse XML received from TCP without any delimiters "). Personally, though, I'd agree - the best "fix" here is to change the data source to not be so pathologically horrible

– Marc Gravell
Mar 27 at 9:01













The OP says : "keep adding received data to string". The xml identification line only occurs once so you can't split on the identification. You have to wait for entire xml file to be received. So you should not be doing the split until entire file is received. The root cause of issue is to make sure you receive the entire message before doing any processing (except joining the received strings).

– jdweng
Mar 27 at 9:16





The OP says : "keep adding received data to string". The xml identification line only occurs once so you can't split on the identification. You have to wait for entire xml file to be received. So you should not be doing the split until entire file is received. The root cause of issue is to make sure you receive the entire message before doing any processing (except joining the received strings).

– jdweng
Mar 27 at 9:16













Maybe I wasn't clear enough (I'll add to post) - I have no control over received data and there are no any kind of delimiters - messages just keep arriving in packets, sometimes whole message (short ACK) or truncated (containing Base64 encoded images) so you need to wait multiple TCP receives to get whole message.

– nighthawk
Mar 27 at 14:20





Maybe I wasn't clear enough (I'll add to post) - I have no control over received data and there are no any kind of delimiters - messages just keep arriving in packets, sometimes whole message (short ACK) or truncated (containing Base64 encoded images) so you need to wait multiple TCP receives to get whole message.

– nighthawk
Mar 27 at 14:20












0






active

oldest

votes










Your Answer






StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













draft saved

draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55372775%2fparsing-xml-from-tcp-connection%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown

























0






active

oldest

votes








0






active

oldest

votes









active

oldest

votes






active

oldest

votes




Is this question similar to what you get asked at work? Learn more about asking and sharing private information with your coworkers using Stack Overflow for Teams.







Is this question similar to what you get asked at work? Learn more about asking and sharing private information with your coworkers using Stack Overflow for Teams.



















draft saved

draft discarded
















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid


  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55372775%2fparsing-xml-from-tcp-connection%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Kamusi Yaliyomo Aina za kamusi | Muundo wa kamusi | Faida za kamusi | Dhima ya picha katika kamusi | Marejeo | Tazama pia | Viungo vya nje | UrambazajiKuhusu kamusiGo-SwahiliWiki-KamusiKamusi ya Kiswahili na Kiingerezakuihariri na kuongeza habari

Swift 4 - func physicsWorld not invoked on collision? The Next CEO of Stack OverflowHow to call Objective-C code from Swift#ifdef replacement in the Swift language@selector() in Swift?#pragma mark in Swift?Swift for loop: for index, element in array?dispatch_after - GCD in Swift?Swift Beta performance: sorting arraysSplit a String into an array in Swift?The use of Swift 3 @objc inference in Swift 4 mode is deprecated?How to optimize UITableViewCell, because my UITableView lags

Access current req object everywhere in Node.js ExpressWhy are global variables considered bad practice? (node.js)Using req & res across functionsHow do I get the path to the current script with Node.js?What is Node.js' Connect, Express and “middleware”?Node.js w/ express error handling in callbackHow to access the GET parameters after “?” in Express?Modify Node.js req object parametersAccess “app” variable inside of ExpressJS/ConnectJS middleware?Node.js Express app - request objectAngular Http Module considered middleware?Session variables in ExpressJSAdd properties to the req object in expressjs with Typescript