This is a pic of my rear door exit handle: https://imgur.com/a/vsJol4i
I am trying to get this tray and handle assembly off the door so I can clean everything up and get to the inside of the door. I have the tray and the handle unscrewed as shown in the pic, but I can't get the handle off. The nut or whatever it is that the handle pivots on takes an Allen wrench but I can't manage to loosen it and it seems I'm in danger of stripping this if I try to lever it too hard. There's also a smaller Allen screw that goes into the side of this handle but I seem to be stripping that without turning it also.
I really don't quite understand how this handle is constructed. Is trying to undo that nut even the correct way to get this handle off?