Using data from over 300,000 visits to an emergency department (ED), we study the accuracy of gatekeeping decisions – the choices that physicians make regarding patient discharge or admission to the hospital. In our study context, we focus specifically on the effectiveness of a second gatekeeping stage in the ED – a clinical decision unit (CDU). While only 9.9% of patients in our sample are routed through the CDU, we find that had the unit not been in place during the observation period, the rates of unnecessary hospitalization and wrongful patient discharge from the ED would have increased by 14.3% and 29.6%, respectively. We also find that the CDU is especially beneficial for patients with a high ex ante risk of experiencing unnecessary hospitalization, with the rate for the most high-risk patients reduced from 14.0% without the CDU to just 4.8% had all such patients been routed through the CDU. The appropriateness of referrals is therefore a key contributor to the CDU’s effectiveness: We estimate that random allocation of patients in our study hospital to the CDU would have reduced the unit’s effectiveness by more than half. Finally, we investigate a critical trade-off in designing a two-stage gatekeeping system: Resources must be split between the two stages, increasing congestion in the first stage when the second stage is enlarged. We demonstrate that in the study hospital, the combination of an ED and CDU performs better than a pooled system that combines the capacity of both stages to enlarge the ED but does not have a designated CDU. In fact, we estimate that in this specific case, reducing the size of the first-stage ED in order to expand CDU capacity from the current 9.9% of ED patients to 25% would further reduce unnecessary hospitalizations by up to 33%. We discuss the insights that these results provide as to the circumstances under which it may be advantageous to add a second stage to a gatekeeping system.