This paper describes an experiment in which an attempt was made to show that Japanese Computer Assisted Language Learning (JCALL) with Animated Graphical Annotation (AGA) is more effective and useful in assisting learners to acquire Japanese language listening skills compared to JCALL without AGA. The independent variable in the experiment is AGA. The principal research question addressed was, “Does the AGA used in the JCALL contribute to the students’ overall pronunciation and listening performances in elementary Japanese language courses by looking into its AKUSENTO (symbol missing), SEION (symbol missing) and DAKUON (symbol missing) and TANON, CHOUON & SOKUON (symbol missing)?” The experiment was conducted in Malaysia last summer. One hundred and twelve students (25 male and 87 female) who were enrolled in Japanese language courses participated in this study. The students were randomly divided into two groups and the pre-test/post-test approach was used in this study. The results of the research showed AGA’s effectiveness in assisting the students to improve their listening skills. However, AGA appeared to be less effective in helping the students to improve their pronunciation skills.