The dynamic behaviors of keyhole and molten pool are growing instable with the increase of welding speed during laser welding. The pores, humping, undercut and other discontinuous weld defects formation are highly related to the instable dynamic behaviors in high speed welding process. Therefore, a 3D numerical simulation method is presented in this paper to analyze the instable dynamic behaviors of keyhole and molten pool in high speed laser welding. The driving forces including surface tension, recoil pressure, thermal buoyancy and thermal boundary are coupling considered in the modeling. The free surfaces of keyhole and molten pool are tracked and the detailed evolution process of the keyhole and molten pool are calculated in the numerical simulation. The oscillation characteristics of keyhole and molten pool during the high speed laser welding process are analyzed and obtained. The results from the numerical simulation are compared with experimental results and good agreement has been found. The influence mechanism of instable dynamic behaviors of keyhole and molten pool on the weld and welding defect formation is investigated based on the simulated results. The results demonstrate that the proposed numerical simulation method can provide the suppression process for welding defects and guide the high efficiency laser welding in practical production.