Graphene composites (GCs) have attracted much attention recently. It is interesting to explore thermal properties of GCs in which graphene filler concentrations are tunable. Here, we use 3ω method to measure the thermal conductivity of GCs synthesized from reduced graphene oxide (RGO) dispersed in polystyrene. To avoid the detrimental effect of lithography processes to GCs, we have developed a novel method employing polyvinyl alcohol and poly(methyl methacrylate) (PMMA) as a holder film to transfer micrometer-sized metal heaters/sensors onto GC surface. Room temperature measurements of the thermal conductivity of GCs are performed. The thermal conductivity is enhanced by ∼ 35 % when adding 5 vol.% of RGO filler concentration. Our measurements will be helpful to probe and understand the thermal transport properties of graphene based composites.