Parsing XML from TCP connectionParsing concatenated, non-delimited XML messages from TCP-stream using C#How do I parse XML from NetworkStream via ReadAsync of XmlReader in C#?How does one parse XML files?Get int value from enum in C#What characters do I need to escape in XML documents?How do I parse XML in Python?What does <![CDATA[]]> in XML mean?Parsing concatenated, non-delimited XML messages from TCP-stream using C#How do you parse and process HTML/XML in PHP?Sending large objects over TCP: “End of Stream encountered before parsing was completed”Sending and Receiving XML data over TCPParsing XML string coming from a TCP/IP connection

Identification of vintage sloping window

How to take the beginning and end parts of a list with simpler syntax?

Why is transplanting a specific intact brain impossible if it is generally possible?

What is the length of pair of wires after twisting them around each other?

On the Rømer experiments and the speed if light

constant evaluation when using differential equations.

Are differences between uniformly distributed numbers uniformly distributed?

How can I solve for the intersection points of two ellipses?

Is it legal for a company to enter an agreement not to hire employees from another company?

As a 16 year old, how can I keep my money safe from my mother?

Who are these characters/superheroes in the posters from Chris's room in Family Guy?

How is this kind of structure made?

Continuous vertical line using booktabs in tabularx table?

Why did the RAAF procure the F/A-18 despite being purpose-built for carriers?

try/finally with bash shell

How should an administrative assistant reply to student addressing them as "Professor" or "Doctor"?

Why isn’t SHA-3 in wider use?

How can Radagast come across Gandalf and Thorin's company?

Tikzpicture - finish drawing a curved line for a cake slice

A simple stop watch which I want to extend

Bitcoin successfully deducted on sender wallet but did not reach receiver wallet

How are you supposed to know the strumming pattern for a song from the "chord sheet music"?

Can you castle with a "ghost" rook?

What are the conventions for transcribing Semitic languages into Greek?



Parsing XML from TCP connection


Parsing concatenated, non-delimited XML messages from TCP-stream using C#How do I parse XML from NetworkStream via ReadAsync of XmlReader in C#?How does one parse XML files?Get int value from enum in C#What characters do I need to escape in XML documents?How do I parse XML in Python?What does <![CDATA[]]> in XML mean?Parsing concatenated, non-delimited XML messages from TCP-stream using C#How do you parse and process HTML/XML in PHP?Sending large objects over TCP: “End of Stream encountered before parsing was completed”Sending and Receiving XML data over TCPParsing XML string coming from a TCP/IP connection






.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;








0















I've run into issue similar to this - I'm trying to parse XML received from TCP without any delimiters and any way of knowing did I receive complete or partial XML. For now, I've taken following approach - keep adding received data to string, try to split it with var xmls = Regex.Split(newData, @"(?=<?xml)"); and try to parse each of strings that came from split - if parsed correctly, it's ok but if not, put the non-complete XML to input buffer and continue. Here is part of receive code that does that:



 string inputBuffer = "";
private void TcpEndReceive(IAsyncResult result)

lock (tcpLock)

int bytesAvailable = 0;
try

bytesAvailable = tcpClient.GetStream().EndRead(result);

catch (Exception ex)

log.Error("TcpEndReceive: error endRead, ", ex);
OnStatusChanged(EReceiverStatus.ConnectionError);
TcpClose();


if (bytesAvailable >= 0)

try

var newData = System.Text.ASCIIEncoding.ASCII.GetString(tcp_data, 0, bytesAvailable);

if (!ParseMessage(newData))

var xmls = Regex.Split(newData, @"(?=<?xml)");
foreach (var xml in xmls)

if (xml.Length == 0) continue;
if (!ParseMessage(xml))

inputBuffer += xml;



if (inputBuffer.Length == 0) return;
if (ParseMessage(inputBuffer))

inputBuffer = "";



catch (Exception ex)

log.Error("TcpEndReceive:error reading data, ", ex);
OnStatusChanged(EReceiverStatus.ConnectionError);
TcpClose();



TcpBeginReceive();



Now, this may look fine but it isn't - if part of message gets out-of-order, this won't work. Anyone has any idea how to solve this? ParseMessage returns true if message is parsed ok. Also, it can happen that you receive part of message, whole message (yay!) or even multiple messages (XMLs) as single transmit.



Any suggestions, ideas or help is welcome. Request is "simple" - receive full or partial XMLs from TCP stream, parse ones that work and wait for other pieces to arrive for those that can't be parsed. Only reliable "delimiter" you can use is <?xml at beginning of every (valid) XML.



[edit] - Additional clarification - I have no control over data that is sent to me - I can't add separators, delimiters or anything. I can only receive strings that may or may be not complete XML. Some messages contain Base64 encoded images and they don't fit in one message - first you get start of XML, then few ms later you get Base64 image (without any indication that more data follows) and then you get end of your XML. As I said, only reliable signal is <?xml at start of valid XML (and beginning of next one, of course). Also, there is a slight problem that there is multiple types of message, with different root elements, so you can't just deserialize to class Message - you have 10-15 different ones. ParseMessage in my example takes the string, finds root element and first node and depending on it deserializes the message in something useful.










share|improve this question


























  • Do not parse xml until the entire TCP message is received. When sending TCP you need to do one of following to make sure receiver know where the end of message occurs 1) ASCII : Terminate message with a character not part of the data 2) ASCII or Binary : Add a byte count to beginning of the message 3) Send Fixed length message

    – jdweng
    Mar 27 at 9:00











  • @jdweng from the question, though, it sounds like the source isn't up to the OP ("I'm trying to parse XML received from TCP without any delimiters "). Personally, though, I'd agree - the best "fix" here is to change the data source to not be so pathologically horrible

    – Marc Gravell
    Mar 27 at 9:01











  • The OP says : "keep adding received data to string". The xml identification line only occurs once so you can't split on the identification. You have to wait for entire xml file to be received. So you should not be doing the split until entire file is received. The root cause of issue is to make sure you receive the entire message before doing any processing (except joining the received strings).

    – jdweng
    Mar 27 at 9:16











  • Maybe I wasn't clear enough (I'll add to post) - I have no control over received data and there are no any kind of delimiters - messages just keep arriving in packets, sometimes whole message (short ACK) or truncated (containing Base64 encoded images) so you need to wait multiple TCP receives to get whole message.

    – nighthawk
    Mar 27 at 14:20

















0















I've run into issue similar to this - I'm trying to parse XML received from TCP without any delimiters and any way of knowing did I receive complete or partial XML. For now, I've taken following approach - keep adding received data to string, try to split it with var xmls = Regex.Split(newData, @"(?=<?xml)"); and try to parse each of strings that came from split - if parsed correctly, it's ok but if not, put the non-complete XML to input buffer and continue. Here is part of receive code that does that:



 string inputBuffer = "";
private void TcpEndReceive(IAsyncResult result)

lock (tcpLock)

int bytesAvailable = 0;
try

bytesAvailable = tcpClient.GetStream().EndRead(result);

catch (Exception ex)

log.Error("TcpEndReceive: error endRead, ", ex);
OnStatusChanged(EReceiverStatus.ConnectionError);
TcpClose();


if (bytesAvailable >= 0)

try

var newData = System.Text.ASCIIEncoding.ASCII.GetString(tcp_data, 0, bytesAvailable);

if (!ParseMessage(newData))

var xmls = Regex.Split(newData, @"(?=<?xml)");
foreach (var xml in xmls)

if (xml.Length == 0) continue;
if (!ParseMessage(xml))

inputBuffer += xml;



if (inputBuffer.Length == 0) return;
if (ParseMessage(inputBuffer))

inputBuffer = "";



catch (Exception ex)

log.Error("TcpEndReceive:error reading data, ", ex);
OnStatusChanged(EReceiverStatus.ConnectionError);
TcpClose();



TcpBeginReceive();



Now, this may look fine but it isn't - if part of message gets out-of-order, this won't work. Anyone has any idea how to solve this? ParseMessage returns true if message is parsed ok. Also, it can happen that you receive part of message, whole message (yay!) or even multiple messages (XMLs) as single transmit.



Any suggestions, ideas or help is welcome. Request is "simple" - receive full or partial XMLs from TCP stream, parse ones that work and wait for other pieces to arrive for those that can't be parsed. Only reliable "delimiter" you can use is <?xml at beginning of every (valid) XML.



[edit] - Additional clarification - I have no control over data that is sent to me - I can't add separators, delimiters or anything. I can only receive strings that may or may be not complete XML. Some messages contain Base64 encoded images and they don't fit in one message - first you get start of XML, then few ms later you get Base64 image (without any indication that more data follows) and then you get end of your XML. As I said, only reliable signal is <?xml at start of valid XML (and beginning of next one, of course). Also, there is a slight problem that there is multiple types of message, with different root elements, so you can't just deserialize to class Message - you have 10-15 different ones. ParseMessage in my example takes the string, finds root element and first node and depending on it deserializes the message in something useful.










share|improve this question


























  • Do not parse xml until the entire TCP message is received. When sending TCP you need to do one of following to make sure receiver know where the end of message occurs 1) ASCII : Terminate message with a character not part of the data 2) ASCII or Binary : Add a byte count to beginning of the message 3) Send Fixed length message

    – jdweng
    Mar 27 at 9:00











  • @jdweng from the question, though, it sounds like the source isn't up to the OP ("I'm trying to parse XML received from TCP without any delimiters "). Personally, though, I'd agree - the best "fix" here is to change the data source to not be so pathologically horrible

    – Marc Gravell
    Mar 27 at 9:01











  • The OP says : "keep adding received data to string". The xml identification line only occurs once so you can't split on the identification. You have to wait for entire xml file to be received. So you should not be doing the split until entire file is received. The root cause of issue is to make sure you receive the entire message before doing any processing (except joining the received strings).

    – jdweng
    Mar 27 at 9:16











  • Maybe I wasn't clear enough (I'll add to post) - I have no control over received data and there are no any kind of delimiters - messages just keep arriving in packets, sometimes whole message (short ACK) or truncated (containing Base64 encoded images) so you need to wait multiple TCP receives to get whole message.

    – nighthawk
    Mar 27 at 14:20













0












0








0








I've run into issue similar to this - I'm trying to parse XML received from TCP without any delimiters and any way of knowing did I receive complete or partial XML. For now, I've taken following approach - keep adding received data to string, try to split it with var xmls = Regex.Split(newData, @"(?=<?xml)"); and try to parse each of strings that came from split - if parsed correctly, it's ok but if not, put the non-complete XML to input buffer and continue. Here is part of receive code that does that:



 string inputBuffer = "";
private void TcpEndReceive(IAsyncResult result)

lock (tcpLock)

int bytesAvailable = 0;
try

bytesAvailable = tcpClient.GetStream().EndRead(result);

catch (Exception ex)

log.Error("TcpEndReceive: error endRead, ", ex);
OnStatusChanged(EReceiverStatus.ConnectionError);
TcpClose();


if (bytesAvailable >= 0)

try

var newData = System.Text.ASCIIEncoding.ASCII.GetString(tcp_data, 0, bytesAvailable);

if (!ParseMessage(newData))

var xmls = Regex.Split(newData, @"(?=<?xml)");
foreach (var xml in xmls)

if (xml.Length == 0) continue;
if (!ParseMessage(xml))

inputBuffer += xml;



if (inputBuffer.Length == 0) return;
if (ParseMessage(inputBuffer))

inputBuffer = "";



catch (Exception ex)

log.Error("TcpEndReceive:error reading data, ", ex);
OnStatusChanged(EReceiverStatus.ConnectionError);
TcpClose();



TcpBeginReceive();



Now, this may look fine but it isn't - if part of message gets out-of-order, this won't work. Anyone has any idea how to solve this? ParseMessage returns true if message is parsed ok. Also, it can happen that you receive part of message, whole message (yay!) or even multiple messages (XMLs) as single transmit.



Any suggestions, ideas or help is welcome. Request is "simple" - receive full or partial XMLs from TCP stream, parse ones that work and wait for other pieces to arrive for those that can't be parsed. Only reliable "delimiter" you can use is <?xml at beginning of every (valid) XML.



[edit] - Additional clarification - I have no control over data that is sent to me - I can't add separators, delimiters or anything. I can only receive strings that may or may be not complete XML. Some messages contain Base64 encoded images and they don't fit in one message - first you get start of XML, then few ms later you get Base64 image (without any indication that more data follows) and then you get end of your XML. As I said, only reliable signal is <?xml at start of valid XML (and beginning of next one, of course). Also, there is a slight problem that there is multiple types of message, with different root elements, so you can't just deserialize to class Message - you have 10-15 different ones. ParseMessage in my example takes the string, finds root element and first node and depending on it deserializes the message in something useful.










share|improve this question
















I've run into issue similar to this - I'm trying to parse XML received from TCP without any delimiters and any way of knowing did I receive complete or partial XML. For now, I've taken following approach - keep adding received data to string, try to split it with var xmls = Regex.Split(newData, @"(?=<?xml)"); and try to parse each of strings that came from split - if parsed correctly, it's ok but if not, put the non-complete XML to input buffer and continue. Here is part of receive code that does that:



 string inputBuffer = "";
private void TcpEndReceive(IAsyncResult result)

lock (tcpLock)

int bytesAvailable = 0;
try

bytesAvailable = tcpClient.GetStream().EndRead(result);

catch (Exception ex)

log.Error("TcpEndReceive: error endRead, ", ex);
OnStatusChanged(EReceiverStatus.ConnectionError);
TcpClose();


if (bytesAvailable >= 0)

try

var newData = System.Text.ASCIIEncoding.ASCII.GetString(tcp_data, 0, bytesAvailable);

if (!ParseMessage(newData))

var xmls = Regex.Split(newData, @"(?=<?xml)");
foreach (var xml in xmls)

if (xml.Length == 0) continue;
if (!ParseMessage(xml))

inputBuffer += xml;



if (inputBuffer.Length == 0) return;
if (ParseMessage(inputBuffer))

inputBuffer = "";



catch (Exception ex)

log.Error("TcpEndReceive:error reading data, ", ex);
OnStatusChanged(EReceiverStatus.ConnectionError);
TcpClose();



TcpBeginReceive();



Now, this may look fine but it isn't - if part of message gets out-of-order, this won't work. Anyone has any idea how to solve this? ParseMessage returns true if message is parsed ok. Also, it can happen that you receive part of message, whole message (yay!) or even multiple messages (XMLs) as single transmit.



Any suggestions, ideas or help is welcome. Request is "simple" - receive full or partial XMLs from TCP stream, parse ones that work and wait for other pieces to arrive for those that can't be parsed. Only reliable "delimiter" you can use is <?xml at beginning of every (valid) XML.



[edit] - Additional clarification - I have no control over data that is sent to me - I can't add separators, delimiters or anything. I can only receive strings that may or may be not complete XML. Some messages contain Base64 encoded images and they don't fit in one message - first you get start of XML, then few ms later you get Base64 image (without any indication that more data follows) and then you get end of your XML. As I said, only reliable signal is <?xml at start of valid XML (and beginning of next one, of course). Also, there is a slight problem that there is multiple types of message, with different root elements, so you can't just deserialize to class Message - you have 10-15 different ones. ParseMessage in my example takes the string, finds root element and first node and depending on it deserializes the message in something useful.







c# xml tcpclient






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Mar 27 at 14:31







nighthawk

















asked Mar 27 at 8:33









nighthawknighthawk

1161 silver badge15 bronze badges




1161 silver badge15 bronze badges















  • Do not parse xml until the entire TCP message is received. When sending TCP you need to do one of following to make sure receiver know where the end of message occurs 1) ASCII : Terminate message with a character not part of the data 2) ASCII or Binary : Add a byte count to beginning of the message 3) Send Fixed length message

    – jdweng
    Mar 27 at 9:00











  • @jdweng from the question, though, it sounds like the source isn't up to the OP ("I'm trying to parse XML received from TCP without any delimiters "). Personally, though, I'd agree - the best "fix" here is to change the data source to not be so pathologically horrible

    – Marc Gravell
    Mar 27 at 9:01











  • The OP says : "keep adding received data to string". The xml identification line only occurs once so you can't split on the identification. You have to wait for entire xml file to be received. So you should not be doing the split until entire file is received. The root cause of issue is to make sure you receive the entire message before doing any processing (except joining the received strings).

    – jdweng
    Mar 27 at 9:16











  • Maybe I wasn't clear enough (I'll add to post) - I have no control over received data and there are no any kind of delimiters - messages just keep arriving in packets, sometimes whole message (short ACK) or truncated (containing Base64 encoded images) so you need to wait multiple TCP receives to get whole message.

    – nighthawk
    Mar 27 at 14:20

















  • Do not parse xml until the entire TCP message is received. When sending TCP you need to do one of following to make sure receiver know where the end of message occurs 1) ASCII : Terminate message with a character not part of the data 2) ASCII or Binary : Add a byte count to beginning of the message 3) Send Fixed length message

    – jdweng
    Mar 27 at 9:00











  • @jdweng from the question, though, it sounds like the source isn't up to the OP ("I'm trying to parse XML received from TCP without any delimiters "). Personally, though, I'd agree - the best "fix" here is to change the data source to not be so pathologically horrible

    – Marc Gravell
    Mar 27 at 9:01











  • The OP says : "keep adding received data to string". The xml identification line only occurs once so you can't split on the identification. You have to wait for entire xml file to be received. So you should not be doing the split until entire file is received. The root cause of issue is to make sure you receive the entire message before doing any processing (except joining the received strings).

    – jdweng
    Mar 27 at 9:16











  • Maybe I wasn't clear enough (I'll add to post) - I have no control over received data and there are no any kind of delimiters - messages just keep arriving in packets, sometimes whole message (short ACK) or truncated (containing Base64 encoded images) so you need to wait multiple TCP receives to get whole message.

    – nighthawk
    Mar 27 at 14:20
















Do not parse xml until the entire TCP message is received. When sending TCP you need to do one of following to make sure receiver know where the end of message occurs 1) ASCII : Terminate message with a character not part of the data 2) ASCII or Binary : Add a byte count to beginning of the message 3) Send Fixed length message

– jdweng
Mar 27 at 9:00





Do not parse xml until the entire TCP message is received. When sending TCP you need to do one of following to make sure receiver know where the end of message occurs 1) ASCII : Terminate message with a character not part of the data 2) ASCII or Binary : Add a byte count to beginning of the message 3) Send Fixed length message

– jdweng
Mar 27 at 9:00













@jdweng from the question, though, it sounds like the source isn't up to the OP ("I'm trying to parse XML received from TCP without any delimiters "). Personally, though, I'd agree - the best "fix" here is to change the data source to not be so pathologically horrible

– Marc Gravell
Mar 27 at 9:01





@jdweng from the question, though, it sounds like the source isn't up to the OP ("I'm trying to parse XML received from TCP without any delimiters "). Personally, though, I'd agree - the best "fix" here is to change the data source to not be so pathologically horrible

– Marc Gravell
Mar 27 at 9:01













The OP says : "keep adding received data to string". The xml identification line only occurs once so you can't split on the identification. You have to wait for entire xml file to be received. So you should not be doing the split until entire file is received. The root cause of issue is to make sure you receive the entire message before doing any processing (except joining the received strings).

– jdweng
Mar 27 at 9:16





The OP says : "keep adding received data to string". The xml identification line only occurs once so you can't split on the identification. You have to wait for entire xml file to be received. So you should not be doing the split until entire file is received. The root cause of issue is to make sure you receive the entire message before doing any processing (except joining the received strings).

– jdweng
Mar 27 at 9:16













Maybe I wasn't clear enough (I'll add to post) - I have no control over received data and there are no any kind of delimiters - messages just keep arriving in packets, sometimes whole message (short ACK) or truncated (containing Base64 encoded images) so you need to wait multiple TCP receives to get whole message.

– nighthawk
Mar 27 at 14:20





Maybe I wasn't clear enough (I'll add to post) - I have no control over received data and there are no any kind of delimiters - messages just keep arriving in packets, sometimes whole message (short ACK) or truncated (containing Base64 encoded images) so you need to wait multiple TCP receives to get whole message.

– nighthawk
Mar 27 at 14:20












0






active

oldest

votes










Your Answer






StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













draft saved

draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55372775%2fparsing-xml-from-tcp-connection%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown

























0






active

oldest

votes








0






active

oldest

votes









active

oldest

votes






active

oldest

votes




Is this question similar to what you get asked at work? Learn more about asking and sharing private information with your coworkers using Stack Overflow for Teams.







Is this question similar to what you get asked at work? Learn more about asking and sharing private information with your coworkers using Stack Overflow for Teams.



















draft saved

draft discarded
















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid


  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55372775%2fparsing-xml-from-tcp-connection%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Kamusi Yaliyomo Aina za kamusi | Muundo wa kamusi | Faida za kamusi | Dhima ya picha katika kamusi | Marejeo | Tazama pia | Viungo vya nje | UrambazajiKuhusu kamusiGo-SwahiliWiki-KamusiKamusi ya Kiswahili na Kiingerezakuihariri na kuongeza habari

SQL error code 1064 with creating Laravel foreign keysForeign key constraints: When to use ON UPDATE and ON DELETEDropping column with foreign key Laravel error: General error: 1025 Error on renameLaravel SQL Can't create tableLaravel Migration foreign key errorLaravel php artisan migrate:refresh giving a syntax errorSQLSTATE[42S01]: Base table or view already exists or Base table or view already exists: 1050 Tableerror in migrating laravel file to xampp serverSyntax error or access violation: 1064:syntax to use near 'unsigned not null, modelName varchar(191) not null, title varchar(191) not nLaravel cannot create new table field in mysqlLaravel 5.7:Last migration creates table but is not registered in the migration table

은진 송씨 목차 역사 본관 분파 인물 조선 왕실과의 인척 관계 집성촌 항렬자 인구 같이 보기 각주 둘러보기 메뉴은진 송씨세종실록 149권, 지리지 충청도 공주목 은진현