Styjun

Please forgive me if I'm unclear, but this is pretty difficult to describe in words. I'm using Ruby for a Rails application to take in values from a CSV file row by row, using Ruby's tohash.select function to generate a hash table of all of the key-value pairs for each row, and then using the create function to generate a table of the rows.

The code works for creating a database table from a CSV, but many records within the CSV have null values for some of the fields/columns. I'd like to have those null values converted to a string like "null" when inserting each row in the CSV to the hash table.

I've tried using a regex to replace the null values with a string, but it hasn't worked. I very well may just be doing it wrong.

require 'csv'

fields = %wlVoterUniqueID sAffNumber szStateVoterID sVoterTitle szNameLast szNameFirst szNameMiddle sNameSuffix sGender szSitusAddress szSitusCity sSitusState sSitusZip sHouseNum sUnitAbbr sUnitNum szStreetName sStreetSuffix sPreDir sPostDir szMailAddress1 szMailAddress2 szMailAddress3 szMailAddress4 szMailZip szPhone szEmailAddress dtBirthDate sBirthPlace dtRegDate dtOrigRegDate dtLastUpdate_dt sStatusCode szStatusReasonDesc sUserCode1 sUserCode2 iDuplicateIDFlag szLanguageName szPartyName szAVStatusAbbr szAVStatusDesc szPrecinctName sPrecinctID sPrecinctPortion sDistrictID_0 iSubDistrict_0 szDistrictName_0 sDistrictID_1 iSubDistrict_1 szDistrictName_1 sDistrictID_2 iSubDistrict_2 szDistrictName_2 sDistrictID_3 iSubDistrict_3 szDistrictName_3 sDistrictID_4 iSubDistrict_4 szDistrictName_4 sDistrictID_5 iSubDistrict_5 szDistrictName_5

if Rails.env.production?
 CSV.foreach(Dir.pwd + "/db/prod.csv", encoding: 'iso-8859-1:utf-8', headers: true) do |row|
 voter_row = row.to_hash.select fields.include?(k)
 Voter.create!(voter_row.to_hash.symbolize_keys)
 end
elsif Rails.env.development?
 CSV.foreach(Dir.pwd + "/db/Cntywd_020819.csv", headers: true) do |row|
 voter_row = row.to_hash.select fields.include?(k)
 Voter.create!(voter_row.to_hash.symbolize_keys)
 end
else
 CSV.foreach(Dir.pwd + "/db/Cntywd_020819.csv", headers: true) do |row|
 voter_row = row.to_hash.select fields.include?(k)
 Voter.create!(voter_row.to_hash.symbolize_keys)
end
end

Wherever I am using row.tohash.select, I'd like to replace null values with an empty string, that way every key in the hash table has a corresponding string ("null" if there is no value).

There is Hash#transform_values method that does the job in a clean and idiomatic way. I'd also suggest using Hash#slice instead of #select:

...
CSV.foreach(Dir.pwd + "/db/prod.csv", encoding: 'iso-8859-1:utf-8', headers: true) do |row|
 attrs = row.to_hash.slice(*fields).transform_values v
 Voter.create!(attrs)
end
...

But to be honest, in practice, I'd propose another solution - using default values for database columns if possible instead of "normalizing" the data on the app level.

You have to iterate over the values and set them where appropriate.

if Rails.env.production?
 CSV.foreach(Dir.pwd + "/db/prod.csv", encoding: 'utf-8', headers: true) do |row|
 voter_row = row.to_hash.select fields.include?(k)
 voter_row.each do |key, value|
 if value.nil?
 voter_row[key] = "null"
 end
 end
 Voter.create!(voter_row.to_hash.symbolize_keys)
 end
else
 CSV.foreach(Dir.pwd + "/db/Cntywd_020819.csv", headers: true) do |row|
 voter_row = row.to_hash.select fields.include?(k)
 voter_row.each do |key, value|
 if value.nil?
 voter_row[key] = "null"
 end
 end
 Voter.create!(voter_row.to_hash.symbolize_keys)
end

I also think your elseif/else is redundant, unless I'm missing something.

This sounds like a job for Hash#transform_values:

h = voter_row.transform_values v

Couple other things:

1. 
   

   You might want to use Hash#slice instead of #select:

   
   
   

   voter_row = row.to_h.slice(*fields)
   

   
   



2. create is happy with string keys so you don't need to call #symbolize_keys.

You can simplify your CSV.foreach blocks to just this:

Voter.create!(row.to_h.slice(*fields))

You could go further and write:

opts = headers: true 
if Rails.env.production?
 csv_file = 'db/prod.csv'
 opts[:encoding] 'iso-8859-1:utf-8'
elsif Rails.env.development?
 csv_file = 'db/Cntywd_020819.csv'
else
 csv_file = 'db/Cntywd_020819.csv'
end
CSV.foreach(Rails.root.join(csv_file), opts) do |row|
 Voter.create!(row.to_h.slice(*fields))
end