That's how it's done! I filled many of my windows using 1/2" plywood laminated with smooth FRP. Then primed the same color as the bus, using a UV blocking automotive primer to protect the FRP. Sealed in place with quality butyl rubber caulking, then ran a bead of exterior caulking around the seam inside, then covered with 1/2' rigid insulation, sealed with sealing tape, and covered with my interior panelling. Works great!