Description
-
Predictions using Human-AI team
This assignment is based on the following research article appeared in 34th edition of Advances in Neural Information Processing Systems (NeurIPS 2021).
Kerrigan, G., Smyth, P., Steyvers, M. (2021). Combining Human Predictions with Model Probabilities via Confusion Matrices and Calibration. Advances in Neural Information Processing Systems, 34.
This paper has been provided as part of this assignment. In this paper, the authors proposed a set of algorithms that combine the probabilistic output of a machine learning model with the human predictions on the same input. This paper shows how the accuracy of predictions are driven by the confidence of machine learning model in its output. Empirical studies of the proposed model have been performed using image classification task with CIFAR-10 and a subset of ImageNet datasets.
The authors have released code for this project here https://github.com/gavinkerrigan/conf_ matrix_and_calibration
You are allowed to use the available code of this paper to complete this assignment.
2
