Styjun

I am getting data as a string from a remote device. I need to parse the data. The data usually come like this:

MO SCGR SC RSITE ALARM_SITUATION
RXOTG-59 59 0 EK0322 ABIS PATH FAULT
RXOCF-59 EK0322 LOCAL MODE
RXOTRX-59-0 4 EK0322 LOCAL MODE
RXOTRX-59-1 EK0322 LOCAL MODE
RXOTRX-59-4 0 EK0322 LOCAL MODE
RXOTRX-59-5 1 3 EK0322 LOCAL MODE
RXOTRX-59-8 EK0322 LOCAL MODE
RXOTRX-59-9 EK0322 LOCAL MODE

I will love to have the data as an array of arrays or any other programmatically sensible structure.

I am splitting the data into an array using:

str.split("rn")

and then removing the extra space on each element in the array with:

tsgs.map! tsg

but this has limitation in that the empty cells are not considered. I expect the array to contain five elements, but it instead contains less than five.

Case 1: In this case, I get the expected result:

RXOTG-59 59 0 EK0322 ABIS PATH FAULT

converts to

["RXOTG-59", "59", "0", "EK0322", "ABIS PATH FAULT"]

Case 2: In this case, I get an unexpected result:

RXOTRX-59-9 EK0322 LOCAL MODE

converts to

["RXOTRX-59-9", "EK0322", "LOCAL MODE"]

 def getCommandResult(tgdatas)
 tgdatas_arr = tgdatas.split("rn")
 tsgs = tgdatas_arr[5..tgdatas_arr.index("END")-2]
 tsgs.map! tsg
 return tsgs
 end

Your string¹, modified slightly:

data = <<END
MO SCGR SC RSITE ALARM_SITUATION
RXOTG-59 59 0 EK0322 ABIS PATH FAULT
RXOCF-59 EK0322 LOCAL MODE
RXOTRX-59-0 4 EK0322 LOCAL MODE
RXOTRX-59-1 EK0322 LOCAL MODE
RXOTRX-59-4 0
RXOTRX-59-5 1 3 EK0322 LOCAL MODE
RXOTRX-59-8 EK0322 LOCAL MODE
RXOTRX-59-9 EK0322 LOCAL MODE
END

This string looks very much like CSV data structure, so we might be tempted to convert it to a CSV string, thereby allowing us to bring to bear the methods provided by the CSV class.

Convert string to CSV string

Code

def convert_to_csv(data)
 cols = data[/.+?n/].gsub(/ S/).map Regexp.last_match.begin(0) 
 data.each_line.map do |s|
 cols.each 
 s.gsub(/ *, */, ',')
 end.join
end

Convert string

Now convert the string data to a CSV string.

csv_data = convert_to_csv(data)

puts csv_data
MO,SCGR,SC,RSITE,ALARM_SITUATION
RXOTG-59,59,0,EK0322,ABIS PATH FAULT
RXOCF-59,,,EK0322,LOCAL MODE
RXOTRX-59-0,4,,EK0322,LOCAL MODE
RXOTRX-59-1,,,EK0322,LOCAL MODE
RXOTRX-59-4,,0
RXOTRX-59-5,1,3,EK0322,LOCAL MODE
RXOTRX-59-8,,,EK0322,LOCAL MODE
RXOTRX-59-9,,,EK0322,LOCAL MODE

Explanation

The steps are as follows.

s = data[/.+?n/]
 #=> "MO SCGR SC RSITE ALARM_SITUATIONn" 
e0 = s.gsub(/ S/)
 #=> #<Enumerator: "MO ... ALARM_SITUATIONn":gsub(/ S/)>
cols = e0.map Regexp.last_match.begin(0) - 1 
 #=> [17, 23, 34, 50] 
e1 = data.each_line
 #=> #<Enumerator: "MO ... LOCAL MODEn":each_line> 
a = e1.map do |s|
 cols.each 
 s.gsub(/ *, */,',')
end
 #=> ["MO,SCGR,SC,RSITE,ALARM_SITUATIONn",
 # "RXOTG-59,59,0,EK0322,ABIS PATH FAULTn",
 # ...
 # "RXOTRX-59-9,,,EK0322,LOCAL MODEn"] 
a.join
 #=> < return value above >

Let's have a closer look at the calculation of a. First, the block variable s is assigned to the first element generated by the enumerator e1:

s = e1.next
 #=> "MO SCGR SC RSITE ALARM_SITUATIONn"

The block calculation is then performed:

cols.each 
s #=> "MO ,SCGR ,SC ,RSITE ,ALARM_SITUATIONn"
s.gsub(/ *, */,',')
 #=> "MO,SCGR,SC,RSITE,ALARM_SITUATIONn"

The regular expression used with gsub reads, "match zero or more spaces followed by a comma, followed by zero or more spaces".

When the short line is passed to the block the following calculation is performed.

s = "RXOTRX-59-4 0"
s.size
 #=> 25
cols
 #=> [17, 23, 34, 50] 
cols.each 
s #=> "RXOTRX-59-4 , ,0" 
s.gsub(/ *, */,',')
 #=> "RXOTRX-59-4,,0" 

The remaining elements of e1 are processed similarly.

Convert the CSV string to a hash

We may now make use of CSV methods. For example, suppose we wish to create an array of hashes whose keys are the header elements, downcased and converted to symbols and values of "SCGR" and "SC" are to be converted to integers. To do that we make use of the class method CSV::new, specifying appropriate values for method options.

Construct the hash

require 'csv'

CSV.new(csv_data, headers: true, header_converters: :symbol,
 converters: :all).to_a.map(&:to_h)
 #=> [:mo=>"RXOTG-59", :scgr=>59, :sc=>0, :rsite=>"EK0322",
 # :alarm_situation=>"ABIS PATH FAULT",
 # :mo=>"RXOCF-59", :scgr=>nil, :sc=>nil, :rsite=>"EK0322",
 # :alarm_situation=>"LOCAL MODE",
 # :mo=>"RXOTRX-59-0", :scgr=>4, :sc=>nil, :rsite=>"EK0322",
 # :alarm_situation=>"LOCAL MODE",
 # :mo=>"RXOTRX-59-1", :scgr=>nil, :sc=>nil, :rsite=>"EK0322",
 # :alarm_situation=>"LOCAL MODE",
 # :mo=>"RXOTRX-59-4", :scgr=>nil, :sc=>0, :rsite=>nil,
 # :alarm_situation=>nil,
 # :mo=>"RXOTRX-59-5", :scgr=>1, :sc=>3, :rsite=>nil"EK0322",
 # :alarm_situation=>"LOCAL MODE",
 # :mo=>"RXOTRX-59-8", :scgr=>nil, :sc=>nil, :rsite=>"EK0322",
 # :alarm_situation=>"LOCAL MODE",
 # :mo=>"RXOTRX-59-9", :scgr=>nil, :sc=>nil, :rsite=>"EK0322",
 # :alarm_situation=>"LOCAL MODE"]

Explanation

The steps are as follows.

csv = CSV.new(csv_data, headers: true, header_converters: :symbol,
 converters: :all)
 #=> <#CSV io_type:StringIO encoding:UTF-8 lineno:0 col_sep:",
 # " row_sep:"n" quote_char:""" headers:true> 
a = csv.to_a
 #=> [#<CSV::Row mo:"RXOTG-59" scgr:59 sc:0 rsite:"EK0322" alarm_situation:"ABIS PATH FAULT">,
 # #<CSV::Row mo:"RXOCF-59" scgr:nil sc:nil rsite:"EK0322" alarm_situation:"LOCAL MODE">,
 # ...
 # #<CSV::Row mo:"RXOTRX-59-9" scgr:nil sc:nil rsite:"EK0322" alarm_situation:"LOCAL MODE">] 
a.map(&:to_h)
 #=> < hash shown above >

^{1 To run the code you will need to un-indent this heredoc (or change the first line to data = <<-END.lines.map(&:lstrip).join).}

String.unpack with directive "A" is nice for fixed width strings.

str = "RXOTRX-59-9 EK0322 LOCAL MODE"
p str.unpack("A20A4A11A16A15" ) # => ["RXOTRX-59-9", "", "", "EK0322", "LOCAL MODE"]

Try if this can be viable for you, given the data_string:

data_string = "MO SCGR SC RSITE ALARM_SITUATIONnRXOTG-59 59 0 EK0322 ABIS PATH FAULTnRXOCF-59 EK0322 LOCAL MODEnRXOTRX-59-0 4 EK0322 LOCAL MODEnRXOTRX-59-1 EK0322 LOCAL MODEnRXOTRX-59-4 0 EK0322 LOCAL MODEnRXOTRX-59-5 1 3 EK0322 LOCAL MODEnRXOTRX-59-8 EK0322 LOCAL MODEnRXOTRX-59-9 EK0322 LOCAL MODE"

Set the starting point of each line, since it seems to be aligned with the header.

data = data_string.split("n")
starts = [0, 18, 24, 35, 51, (data.map(&:size)).max ]

Then map each line considering the starting points, stripping trailing spaces:

data = data.map starts.each_cons(2).map a,b 

So you'll end up with this array:

# [["MO", "SCGR", "SC", "RSITE", "ALARM_SITUATION"]
# ["RXOTG-59", "59", "0", "EK0322", "ABIS PATH FAULT"]
# ["RXOCF-59", "", "", "EK0322", "LOCAL MODE"]
# ["RXOTRX-59-0", "4", "", "EK0322", "LOCAL MODE"]
# ["RXOTRX-59-1", "", "", "EK0322", "LOCAL MODE"]
# ["RXOTRX-59-4", "", "0", "EK0322", "LOCAL MODE"]
# ["RXOTRX-59-5", "1", "3", "EK0322", "LOCAL MODE"]
# ["RXOTRX-59-8", "", "", "EK0322", "LOCAL MODE"]
# ["RXOTRX-59-9", "", "", "EK0322", "LOCAL MODE"]]

You can then convert it to a hash or use the csv library to manipulate your data.

headers = data[0]
body = data[1..]

body.map headers.map(&:to_sym).zip(line).to_h 
#=> [:MO=>"RXOTG-59", :SCGR=>"59", :SC=>"0", :RSITE=>"EK0322", :ALARM_SITUATION=>"ABIS PATH FAULT", :MO=>"RXOCF-59", :SCGR=>"", :SC=>"", :RSITE=>"EK0322", :ALARM_SITUATION=>"LOCAL MODE", ...