Styjun

I have a (123, 3072) array and i need to split it to 5 approximately same folds (approximately, because 123 can't be divided by 5, for example) in order to do a 5-fold cross-validation. scikit-learn is not allowed. I tried to get 2 ndarrays of size (3, 25, 3072) and (2, 24, 3072). Now i need to combine them, but every function i try raises this:

ValueError: all the input array dimensions except for the concatenation 
axis must match exactly 

Is it possible to concatenate them?

This is my code:

num_folds = 5
mod = binary_train_X.shape[0] % num_folds
first_records = (binary_train_X.shape[0] - mod) // num_folds + 1
last_records = first_records - 1
first_part = binary_train_X[:mod * first_records].reshape([mod, first_records, -1])
second_part = binary_train_X[mod * first_records:].reshape([num_folds - mod, last_records, -1])
folds_X = np.concatenate((first_part, second_part))

Or maybe there is another way to split it into 5 parts(folds)?

Something very similar to this, if not exactly this.

def k_fold(array, num_folds): #New to WOS
 #Splits along axis 0 of array
 folds = []
 start = 0
 step = array.shape[0]/num_folds
 for i in range(num_folds):
 end = int(start + step)
 start = int(start)
 fold = array[start:end]
 rest_of_array = np.concatenate((array[:start],array[end:]), axis = 0)
 start = end
 folds.append((fold, rest_of_array))
 return folds

Since 377856 (123*3072) is not divisible by 15360 (5*3072) (123 is not divisible by 5), it is only possible to create 5 equal slices of 3072 by either truncating or padding to a multiple of 15360 (5*3072).

Truncating creates shape (5, 24, 3072) by discarding values from the end until it aligns:

folds = binary_train_X.flatten()[:np.prod(binary_train_X.shape)//(5*3072)*(5*3072)].reshape(5, -1, 3072)
# this discards 9216 (3072*3) values

Padding creates shape (5, 25, 3072) by appending zeros to the end until it aligns:

folds = np.pad(binary_train_X.flatten(), (0, -(-np.prod(binary_train_X.shape)//(5*3072))*(5*3072)-np.prod(binary_train_X.shape)), 'constant').reshape(5, -1, 3072)
# this appends 6144 (3072*2) zeros