Well done in clearing the cavity
I found it easier to remove one corner brick on the outside, and rake the rubbish towards me. A builders canister vac will suck out the loose stuff, you can buy additional metal extension tubes to increase your reach, and poked into sand and mortar, they will break it up enough to suck it out.
You can get a garden tool with a long metal handle and a claw on the end, I think it is intended for disabled people to do weeding. There may be better tools and techniques others have used.