Styjun

I am trying to use the boxcoxnc function in the AID package to calculate normalized data using the Shapiro-Wilcox W statistic to determine lambda.

I want the boxcoxnc function to run on each column in my data frame in a for loop.

data<-data.frame(data[,2:27])

for (f in 1:length(data))
 model<-boxcoxnc(as.matrix(as.numeric(unlist(data[f]))),
 method="sw",lambda = as.numeric(seq(-20,20,0.01)))

The first three columns work fine and when I get to the fourth I get the error:

Error in boxcoxnc(as.matrix(as.numeric(unlist(data[f]))), method = "sw", : 
 Enlarge the range of the lambda

Which I do, enlarge the range of lambda to(-21, -20, 0.01) and then get the following error on the first column.

Error in shapiro.test(store2[[x]]) : all 'x' values are identical

However, the data is not identical. It is only certain columns in my data frame that does this and I do not know why. The fourth column that calls the first error is this:
1.539
1.587
1.558
1.625
1.651
1.659
1.654
1.643
1.53
1.552
1.537
1.522
1.559
1.636
1.57
1.631
1.544
1.625
1.552
1.519
1.556
1.528
1.616
1.554
1.571
1.534
1.574
1.578
1.574
1.533
1.54
1.531
1.561
1.576
1.624
1.593
1.557
1.556
1.559
1.59

The first column is this: 6.301
6.611
6.448
7.049
7.068
7.208
7.215
7.084
6.129
6.471
6.295
5.984
6.34
7.052
6.448
6.885
6.42
6.963
6.169
6.185
6.289
6.05
6.901
6.333
6.458
6.228
6.458
6.477
6.71
6.296
6.147
6.171
6.278
6.667
6.932
6.646
6.369
6.408
6.466
6.688

Any help is really appreciated.

One thing you must keep always in mind in R is that by design R tends to just to keep going no matter what, coercing and converting the data even when some of the transformation actually do not make sense. This results in values and errors messages down the line that do not make sense and that is what may be happening here. If you inspect the result of as.matrix(as.numeric(...)), for every one of those columns I likely that is not what you expect it to be.

Without knowing exactly how boxcoxnc works I suggest the following alternative code to make it more readable and may-be even fix the bug but that is a big maybe:

for (col in 2:27) 
 model <- boxcoxnc(data[,col], method="sw", lambda = seq(-20,20,0.01))
 # what are you trying to do with model here, it is rewritten every time.

Comments:

1. subsetting the original data is unnecessary since you are iterating through the columns by index.




2. even when data[col] would work (since a data.frame is in fact a list of columns) is more appropriate to do data[,col]. Also instead of length(data) you should write ncol(data) but that expression is gone anyway.




3. as.matrix(as.numeric(unlist(...))) seems totally unnecessary here and is just a chance to something go wrong in terms of R doing an untended conversion. Perhaps as.numeric is necessary iff boxcoxnc is a bit particular and really cannot accept anything but a numeric vector.




4. as.numeric(seq(...)) can be just sec(...); it would be surprising if seq would return anything but a numeric vector.

Now, something you should consider is perhaps some of those columns do not contain numeric data. If it say numbers but as strings then yes you need as.numeric. Can you confirm that there is no column that contains anything but numerics and integers typed data? Strings or factors would be problematic and might be the root cause of your issues. What is the result of:

sapply(d, class)

Btw apply methods are preferable to for loops so perhaps you want to go that route perhaps you co do something like this:

models <- sapply(data[,2:27], function(col) 
 boxcoxnc(col, method="sw", lambda = seq(-20,20,0.01)) 
)