(4) is the correct answer.

     Because:
     (x3-3xy+y2)(4xy2) = 4x4y2-12x2y3+4xy4