How to remove handles on sliding patio doors

[point5clue]

Hi,
The handles on both sides on my patio doors are starting to come loose. Just a slight wobble at this stage, but with heavy door it will only get worse.
I'm fairly sure they just connect with a bolt to the inside of the plate, but I can't get the whole assembly off the door to tighten them.

There are two long bolts from the inside, and they come out easily. The handle now seems to be held together by the locking lever either side.

I've tried gripping the lever with pliers softened with a bit of inner tube and pulling, but I'm reluctant to put too much force in to it.

There's no grub screw so I thought I'd ask if anyone knows the best way to do this before I break it trying to

 

[point5clue]

Well, after speaking to someone this morning I decided to have another go. I shut the door fully first so it was braced in the frame and gave the handle a jolly hard pull and it came off. Outside face came off too, and bounced off the patio, but only a tiny chip

A quick tighten of the screws and its all back to solid.