Are you familar with the old fashioned storm windows? I'm referring to the single pane of glass/plastic that is placed on the outside of a house and covers the whole window in the wintertime to keep the cold out creating a "dual pane" window. You can do the same with your lovely little windows.
Frame in around your windows, on the inside
, with PVC "wood" making sure to cover all the metal parts. The PVC will not rot or discolour from possible moisture plus it will provide a good thermal break. Either route a rabbet (a little step down) into the trim to accept a piece of plexiglass or use another low profile piece of PVC trim to edge around the plexi. Use a bead of siliconized latex adhesive caulk (I prefer the ones that say "elastometric" for this type of use) to seal to the window trim along with a few screws (now you have a thermal bridge and must cover the head of the screws to break the thermal bridge). You might want to drill a couple of tiny holes thru the plexi/PVC and run a tiny screw into the PVC material but not touching the metal of the bus (in case you don't trust the caulk to hold the plexi to the PVC... I don't or if you want to be able to remove them later, in which case use a length of vinyl weatherstripping on the trim along with the screws). If the screw touches the metal of the bus/frame it will continue the thermal bridge. You would need to cap the screw head from the interior some how if that happens, easier to use a short tiny screw in the first place and be careful. If you plan on prettying up the windows with a window film, I would apply that first if you plan on caulking the plexi in. If using the screws & weatherstripping, you can add it at anytime. (note: make sure your windows and the plexi are VERY clean as you can't get them apart, without much difficulty, to clean once glued up). I prefer the weatherstripping/screws version because I know I would change my mind at some point and have to tear the things apart to change what is behind the plexi.
The PVC "wood" can be securely glued together using clear PVC plumbing pipe glue (don't get sloppy with it). It can also be painted (I suggest either a paint made for painting plastic or use a clear matte version as a primer that you can paint over with whatever you choose) or stained with a gel stain. As well as shaped with a router. You can attach PVC "wood" with screws (then plug them) or glue them to the structure using an adhesive or adhesive caulk. We will screw/caulk for our windows and then cover the screw heads with another layer of PVC "wood" glued to the bottom layer (it's a stacking of profiles to get the look we want). But for the decorative thermal breaks on the ceiling (primed with gray tinted Kilz2 then painted with metallic paint), we just used PL400 Subfloor & Deck Adhesive
and prop sticks until the adhesive dried completely.