I hadn't considered whether a freezer might be built with better insulation than a refrigerator is. Without challenging your assertion that "freezers generally have better insulation" -- well, ok, I'm challenging it but in a friendly way
-- I wouldn't make the assumption that freezers are better just because they have to be colder inside. So, uncertainty about the basic premise aside..
The shelves could get pretty cold. That might cause frost damage to delicate foods that wouldn't ordinarily be damaged in a forced-air fridge with the same air temperature setpoint. I'm thinking of produce, maybe some dairy, etc. These things might have to be put on an insulator to protect them from the cold shelf.
What about the location of the condenser? The two upright freezers I've owned both had their condenser buried under the skin of the appliance, whereas the refrigerators I've owned had an exposed condenser mounted to the back or under the bottom, sometimes with a fan to help them shed heat. Depending on how you built around the unit one condenser style/location or the other might be preferable (or entirely unworkable).
I have no idea at what point the efficiency improvement of better wall insulation, whether a better-built unit or added by the user, is dwarfed by the number and duration of door-open air changes per day.