How to predict with missing data column in test dataset

[alexmilanov]

Hello! I'm using Ludwig v.0.5.3 and I'm experimenting with Titanic dataset. First, I train my model with the data which contain the following columns:
PassengerId,Survived,Pclass,Name,Sex,Age,SibSp,Parch,Ticket,Fare,Cabin,Embarked
Once my model is trained, I'm invoking the following python code which is trying to predict the outcome for given Titanic passenger:
self.model.predict(dataset=self.testFilePath), where self.model is the aforementioned trained Ludwig model.
The problem is that the test dataset is missing the Survived column:
PassengerId,Pclass,Name,Sex,Age,SibSp,Parch,Ticket,Fare,Cabin,Embarked, so Ludwig is throwing error that this column is missing.
Here is my model description:
{"input_features": [{"name": "Pclass", "type": "category"}, {"name": "Sex", "type": "category"}, {"name": "Age", "type": "number", "preprocessing": {"missing_value_strategy": "fill_with_mean"}}, {"name": "SibSp", "type": "number"}, {"name": "Parch", "type": "number"}, {"name": "Fare", "type": "number", "preprocessing": {"missing_value_strategy": "fill_with_mean"}}, {"name": "Embarked", "type": "category"}], "output_features": [{"name": "Survived", "type": "binary"}]}

My questions is how do I can use predict method similarly as with the --only_predict parameter, so it can predict the outcome without evaluating the model performance? Is it possible at all?

[connor-mccorm]

Hi @alexmilanov, could you possibly post the command you are running and the error message you are experiencing so I can take a look and see what's going wrong here?

[connor-mccorm]

Perfect, thank you. Taking a look right now and will get back to you ASAP.

[alexmilanov]

Noted, thanks!

[alexmilanov]

Sorry, it look like that it is bug in my code, I was calling model.evaluate() instead model.predict(). Everything work just fine. Thanks for the help!

        Survived_probabilities  Survived_predictions  ...  Survived_probabilities_True  Survived_probability
0       [0.9469244, 0.0530756]                 False  ...                     0.053076              0.946924
1     [0.79066443, 0.20933555]                 False  ...                     0.209336              0.790664
2    [0.99010724, 0.009892752]                 False  ...                     0.009893              0.990107
3     [0.82891405, 0.17108598]                 False  ...                     0.171086              0.828914
4     [0.30545247, 0.69454753]                  True  ...                     0.694548              0.694548
..                         ...                   ...  ...                          ...                   ...
413    [0.8645514, 0.13544856]                 False  ...                     0.135449              0.864551
414   [0.078442276, 0.9215577]                  True  ...                     0.921558              0.921558
415    [0.9402634, 0.05973662]                 False  ...                     0.059737              0.940263
416    [0.8645514, 0.13544856]                 False  ...                     0.135449              0.864551
417   [0.89675194, 0.10324805]                 False  ...                     0.103248              0.896752

[connor-mccorm]

So at first glance I noticed something, though I don't think this is what is causing the error here since the actual error is coming from the evaluate command being called. I acknowledge that this is odd since I can see that you are calling self.model.predict() within your predict method posted above, but just to cover all our bases: looking at your code, if looks like the initializer to your BinaryClassification class takes the arguments in the order trainFilePath and then testFilePath, but when you initialize the class below, it looks like you're passing in 'titanic/text.csv' (also wanted to check that this wasn't a type for test.csv just in case) into the trainFilePath spot and passing in 'titanic/train.csv' into the testFilePath spot.

Looking at the stack trace however, it looks like at line 20 of file TaskFactory.py, bc.evaluate is being called here when it should be bc. predict(). Since I can't see all of the code, I'm not sure exactly the nature of how this is being called, however what I can tell you is that barring any issues caused by the mismatched file loading here, this code should be able to predict on the test set without the target column present. When using the predict() command from our API, it only produces predictions, however the evaluate() command produces predictions and evaluates them thus requiring the target column. So if you're comfortable with posting more code here, I'm happy to help you debug further. Otherwise, I would point you to line 20 of TaskFactory.py and line 27 of BinaryClassification.py to try and find where your code is calling .evaluate instead of .predict().

[connor-mccorm]

Happy to help!