Search for b -> u transitions in B- -> DK- and D*K- decays

We report results from an updated study of the suppressed decays B-→DK- and B-→D*K- followed by D→K+π-, where D(*) indicates a D(*)0 or a D̅ (*)0 meson, and D*→Dπ0 or D*→Dγ. These decays are sensitive to the Cabibbo-Kobayashi-Maskawa unitarity triangle angle γ due to interference between the b→c transition B-→D(*)0K- followed by the doubly Cabibbo-suppressed decay D0→K+π-, and the b→u transition B-→D̅ (*)0K- followed by the Cabibbo-favored decay D̅ 0→K+π-. We also report an analysis of the decay B-→D(*)π- with the D decaying into the doubly Cabibbo-suppressed mode D→K+π-. Our results are based on 467× 10^6 Υ(4S)→BB̅ decays collected with the BABAR detector at SLAC. We measure the ratios R(*) of the suppressed ([K+π-]DK-/π-) to favored ([K-π+]DK-/π-) branching fractions as well as the CP asymmetries A(*) of those modes. We see indications of signals for the B-→DK- and B-→DDπ0*K- suppressed modes, with statistical significances of 2.1 and 2.2σ, respectively, and we measure: RDK=(1.1±0.6±0.2)×10-2, ADK=-0.86±0.47-0.16+0.12, R(Dπ0)K*=(1.8±0.9± 0.4)×10-2, A(Dπ0)K*=+0.77±0.35±0.12, R(Dγ)K*=(1.3±1.4±0.8)×10-2,A(Dγ)K*=+0.36 ±0.94-0.41+0.25, where the first uncertainty is statistical and the second is systematic. We use a frequentist approach to obtain the magnitude of the ratio rB≡|A(B-→D̅ 0K-)/A(B-→D0K-)|=(9.5-4.1+5.1)%, with rB<16.7% at 90% confidence level. In the case of B-→D*K- we find rB*≡|A(B-→D̅ *0K-)/A(B-→D*0K-)|=(9.6-5.1+3.5)%, with rB*<15.0% at 90% confidence level.