Members-Only
Recent Talks & Demos are for members only
You must be an AI Tinkerers active member to view these talks and demos.
Reliable ML Predictions with Conformal Prediction
Learn how to make ML predictions reliable with Conformal Prediction. This model-free framework provides statistically valid confidence guarantees, ensuring your models say "I don't know" when unsure.
ML models make prediction but these predictions can be wrong. In high-stakes environments like healthcare, this has consequences. In normal everyday tasks like search or text generation, these wrong predictions are known as hallucinations. To integrate and augment ML models in society, we need them to be reliable!
In this demo, I will show how to implement Conformal Prediction, a model-free framework that reasons how confident our model is. Ideally, if our model is not confident then, they should say “I don’t know”. However, current model deployments don’t do this.
Conformal Prediction wraps around any model to output a set instead of a single point. The sets are guaranteed to contain the true label with statistical validity. With this, we can statistically guarantee how reliable our predictions are instead of using adhoc unreliable methods that have no guarantees.
I will be demo’ing how to convert your model to be more reliable with Conformal Prediction. This framework is general and useful in many applications. This will be a live demo that goes through the coding process and technical implementation of Conformal Prediction.