With Diesel engines, fuel milage is (or at least seems to me) more dependent of the load than is the case with gasoline engines. I can get 10 MPG with a 14 Liter 18-wheeler engine -- when I'm running empty. And even better if I'm not pulling a trailer at all.
So I'm in favor of big engines. Climbing hills at 20 MPH stinks.