brainome logo

302 Generating Probabilities

Brainome’s Random Forest and Neural Network model predictors can also generate probabilities.

Prerequisites

This notebook assumes brainome is installed as per notebook brainome_101_Quick_Start

The data sets are:

Predictors require numpy to run and optionally scipy to generate a confusion matrix.

!python3 -m pip install brainome --quiet
!brainome --version

import urllib.request as request
response2 = request.urlretrieve('https://download.brainome.ai/data/public/titanic_predict.csv', 'titanic_predict.csv')
%ls -lh titanic_predict.csv
brainome v1.006-19-prod
-rw-r--r-- 1 runner docker 858 Oct 20 18:09 titanic_predict.csv

Generate a predictor

The predictor filename is predictor_302.py

!brainome https://download.brainome.ai/data/public/titanic_train.csv -y -o predictor_302.py -modelonly -q
print('\nCreated predictor_302.py')
!ls -lh predictor_302.py
WARNING: Could not detect a GPU. Neural Network generation will be slow.
Created predictor_302.py
-rw-r--r-- 1 runner docker 39K Oct 20 18:09 predictor_302.py

Generating classification probabilities for a data set

Rather than picking a single class, this feature outputs the probabilities for each class.

# using pandas to read csv data
%pip install pandas
import pandas as pd
import predictor_302 as predictor
# reading csv file
predict_data = pd.read_csv('titanic_predict.csv', na_values=[], na_filter=False)
# REQUIRED: strip the headers from dataset
predict_values = predict_data.values
probabilities_output = predictor.predict(predict_values, return_probabilities=True)
print(' Prediction Probabilities '.center(80, '-'))
print(probabilities_output)
Requirement already satisfied: pandas in /opt/hostedtoolcache/Python/3.9.7/x64/lib/python3.9/site-packages (1.3.4)
Requirement already satisfied: python-dateutil>=2.7.3 in /opt/hostedtoolcache/Python/3.9.7/x64/lib/python3.9/site-packages (from pandas) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /opt/hostedtoolcache/Python/3.9.7/x64/lib/python3.9/site-packages (from pandas) (2021.3)
Requirement already satisfied: numpy>=1.17.3 in /opt/hostedtoolcache/Python/3.9.7/x64/lib/python3.9/site-packages (from pandas) (1.21.2)
Requirement already satisfied: six>=1.5 in /opt/hostedtoolcache/Python/3.9.7/x64/lib/python3.9/site-packages (from python-dateutil>=2.7.3->pandas) (1.16.0)
Note: you may need to restart the kernel to use updated packages.
--------------------------- Prediction Probabilities ---------------------------
[['died' 'survived']
 ['0.10347254522374348' '0.8965274547762565']
 ['0.8357493590851747' '0.16425064091482533']
 ['0.5723091317715159' '0.4276908682284841']
 ['0.8357493590851747' '0.16425064091482533']
 ['0.8833638515749265' '0.11663614842507353']
 ['0.8204901964380503' '0.17950980356194968']
 ['0.8215620538923634' '0.17843794610763664']
 ['0.10347254522374348' '0.8965274547762565']
 ['0.42322660063987716' '0.5767733993601228']
 ['0.7495210975955352' '0.2504789024044648']
 ['0.8833638515749265' '0.11663614842507353']]

Combining probabilities into the source data set

import numpy as np
predict_header = predict_data.columns.values
full_output = np.concatenate((
    np.concatenate((predict_header.reshape(1, -1), predict_data)), probabilities_output), axis=1)
pd.DataFrame(full_output)
0 1 2 3 4 5 6 7 8 9 10 11 12
0 PassengerId Cabin_Class Name Sex Age Sibling_Spouse Parent_Children Ticket_Number Fare Cabin_Number Port_of_Embarkation died survived
1 881 2 Shelley, Mrs. William (Imanita Parrish Hall) female 25 0 1 230433 26.0 S 0.10347254522374348 0.8965274547762565
2 882 3 Markun, Mr. Johann male 33 0 0 349257 7.8958 S 0.8357493590851747 0.16425064091482533
3 883 3 Dahlberg, Miss. Gerda Ulrika female 22 0 0 7552 10.5167 S 0.5723091317715159 0.4276908682284841
4 884 2 Banfield, Mr. Frederick James male 28 0 0 C.A./SOTON 34068 10.5 S 0.8357493590851747 0.16425064091482533
5 885 3 Sutehall, Mr. Henry Jr male 25 0 0 SOTON/OQ 392076 7.05 S 0.8833638515749265 0.11663614842507353
6 886 3 Rice, Mrs. William (Margaret Norton) female 39 0 5 382652 29.125 Q 0.8204901964380503 0.17950980356194968
7 887 2 Montvila, Rev. Juozas male 27 0 0 211536 13.0 S 0.8215620538923634 0.17843794610763664
8 888 1 Graham, Miss. Margaret Edith female 19 0 0 112053 30.0 B42 S 0.10347254522374348 0.8965274547762565
9 889 3 Johnston, Miss. Catherine Helen Carrie"" female 1 2 W./C. 6607 23.45 S 0.42322660063987716 0.5767733993601228
10 890 1 Behr, Mr. Karl Howell male 26 0 0 111369 30.0 C148 C 0.7495210975955352 0.2504789024044648
11 891 3 Dooley, Mr. Patrick male 32 0 0 370376 7.75 Q 0.8833638515749265 0.11663614842507353