Styjun

When using the R package RecordLinkage, some of the outputs following epiClassify() or emClassify() functions (which can operate on RLBigDataLinkageclass objects) will output errors. These errors are not seen when I use the functions designed for smaller data comparisons, such as compare.linkage(). The documentation describing this can be found in the package vignettes here.

My overall objective is to 'fuzzy match' data between two tables; once I know which rows are similar between the tables based on other variables, I can then use the index to grab unique IDs from the column in one table which are missing from the other table.

Data: This is some dummy data to reproduce the errors...

library(tibble)
table1 <- tibble(col1 = c("JIMMY", "SARA", "AYIL", "JIM", "JOHN"),
 col2 = c("OHEARN", "HANDLE", "HASE", "JHORN", ""),
 col3 = c("jdt322", "jdb122", "", "ddd532", "ddd444"))

table2 <- tibble(col1 = c("JIMMY", "SARAH", "SARA", "AYIL", "JIM", "JOHN", "timm"),
 col2 = c("OHEARN", "HAND","H", "HASE", "JORN", "", ""),
 col3 = c("jdt322", "jda122", "jdb112", "", "ddd532", "ddd444", "ddd322"))

When I use the above data in the functions for smaller comparisons, it works fine, without errors:

mypairs <- RecordLinkage::compare.linkage(table1, table2, strcmp = T)
mypairs_weights <- emWeights(mypairs)
result <- emClassify(mypairs_weights, threshold.lower = 1)
getTable(result)
summary(result)
getPairs(result, min.weight = 1)

However, after I use the RLBigData class (as in the code below), I get errors when:

• I try to access the object result after emClassify(): Error in nrow(object@pairs) :
  no slot of name "pairs" for this object of class "RLResult"




• Trying to grab the summary with summary(result): Error in dbGetQuery(object@con, "select count() from data1") :
  no slot of name "con" for this object of class "RLBigDataLinkage"*




• Trying to grab the comparison table with getTable(result): Error in table.ff(object@data@pairs$is_match, object@prediction, useNA = "ifany") :
  Only vmodes integer currently allowed - are you sure ... contains only factors or integers?




• Warnings are also thrown when first running the RLBigDataLinkage() function: Warning messages:
  1: In result_fetch(res@ptr, n = n) :
  Don't need to call dbFetch() for statements, only for queries

The following code should reproduce these errors

mypairs <- RLBigDataLinkage(table1, table2,
 strcmp = T,
 strcmpfun = "jarowinkler")

mypairs_weights <- emWeights(mypairs)
result <- emClassify(mypairs_weights, 0.6)
result
getTable(result)
summary(result)
getPairs(result, min.weight = 0.5)

I am not certain as to why the code spits these errors in the second case. I wanted to troubleshoot the code on a small dataset before moving onto larger datasets. I would greatly appreciate it if anyone could shed light on these errors and warnings output by this package.