TEQnation 2021 – Nadia Alshahwan – Improving the Quality of Apps at Facebook with Virtual Alpha

Apps use Alpha and Beta programs to test changes on a smaller scale before releasing to the wider user population. This helps limit the impact of code quality issues on the wider population. However, Alpha and Beta programs are still subjecting real users to crashes and performance issues which is something we always want to avoid. We also do not have control over how quick after the release our Alpha or Beta users use the app and which parts they cover. At Facebook, we are trying to solve this problem by releasing our apps pre Alpha to a simulation of production that we call Virtual Alpha. This is a realistic population of virtual users that are operated by algorithms that attempt to imitate real users. Using Virtual Alpha we can not only find realistic crashes but also predict their impact on real users by using the impact on our virtual population as a proxy. This allowed us to block many high impact crashes before a single real user is impacted.