Learning

Friday, February 21, 2014 9:58 AM Release criterion plays a critical role in releasing software. Good release criterion includes setting up the right quality bars that product has to pass before it's deemed ready. Traditional software testing methods focused on various test design approaches on coming up with the right set of test cases that assures some measure of quality. How does this work in a world where the software is largely machine learned, how does one come up with a good release criterion.? When shipping initial set of models where models are only trained with limited set of real world data it's hard to estimate how well they do once the models are live and taking the real traffic.