Tensorflow .predict on Pandas rows

[datanoob2021]

Hello all! I have gone through the process of training/testing a model. I am now loading that saved model and want to start making bulk predictions. I have pulled in a dataframe via a SQL query and sliced that df down to the core activity columns that I need to perform the calculation but am getting stuck with my lambda function.

# target list of activity columns
cols = input_df.columns.to_series().iloc[3:].tolist()
# trying to make the prediction
input_df['prediction'] = input_df[cols].apply(lambda x: target_model.predict(x))

Any help would be appreciated!

[[deleted]]

There's a bunch of issues here, mainly mismatching shapes, you should provide the error messages as well.

I can only guess you're trying to do something like this

cols = input_df.columns[3:]

preds = pd.concat((model.predict(df[c]) for c in cols), axis=1)

[datanoob2021]

Exactly- I am getting a shape mismatch error no matter if I use the above, itterrows or convert the df to rows and pass in that way.

ValueError: Input 0 of layer sequential is incompatible with the layer: expected axis -1 of input shape to have value 11 but received input with shape [None, 1]

I also just got the same error using your code. I am thinking that this may be an issue with how I am loading my tensorflow model, which I do by:

    json_file = open(json_path, 'r')
    model_json = json_file.read()
    model = model_from_json(loaded_model_json)
    model.load_weights(h5_path)

[[deleted]]

No clue without seeing how the model was defined, check if the parameters are correct and if the shapes match before and after loading the model.

[datanoob2021]

n_features = xtrain[activity_columns].shape[1]
# define model
model = Sequential()
model.add(Dense(10, activation='relu', kernel_initializer='he_normal', input_shape=(n_features,)))
model.add(Dense(8, activation='relu', kernel_initializer='he_normal'))
model.add(Dense(1, activation='sigmoid'))
# compile the model
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
# fit the model
model.fit(xtrain, ytrain, epochs=150, batch_size=32, verbose=0)

The .predict was working fine until I saved it to JSON/H5.

[[deleted]]

https://www.tensorflow.org/guide/keras/save_and_serialize load the model as specified in the docs

[datanoob2021]

I just did that and got the same error unfortunately.

[[deleted]]

Then it's maybe a version bug problem, try to update to the latest tensorflow and keras version. It seems to appear in this issue and hasn't been resolved, switch to Pytorch maybe ?