Had the same situation with my 94 Dodge Dakota. It's about a half mile to the blacktop from my driveway. Rears would lock up on the dirt for the first 2 or 3 stops. Never found any leaks or harware issues, tried adjusting. This was 5 yrs ago and the stock springs and hardware are still in place. The problem mysteriously dissappeared. I did notice it once or twice on my Camaro also, both kept in the same garage. I did change brakes also due to wear when I first bought the truck but before the lock up issue. I'm thinking brake lining material? Stock drum shoes are usually a light tan or charcoal color. Not sure of the materials that make either of these up but it might be a starting point. Match replacements to original equipment material.
My 86 Camaro had crappy brakes also, wouldn't lock up no matter what. I changed the front rubber lines and this nearly doubled in stopping power. The insides of the hose can break down and block fluid flow. After this the LH front would lock but not the right. Changed the RH caliper and that also made a huge difference in stopping power. Wasn't frozen but seemed tighter than the left side. After this, both would lock but not as hard as just the LH side before the repair. I then got into the proportioning valve and wanted to mod this as sugested on various internet sites. Took mine apart to investigate and found the proportioniong section was full of debris. Cleaned it out and reassembled it, unmodified, and the brakes were better than just about any car I ever drove with front disc, RR drum brakes. (Note if you dissasmble this note the position of internal parts carefully, there is a rubber cup shaped piece that if installed backwards will prevent all oil flow to the rear brakes, ask me how I know!) I cleaned mine out on the car by removing the end cap on the front of the prop. valve. Remove the cap, spring, spool, and rubber cup, allow fluid to gravity flow out the opening, rinsing out the debris. Reinstall the parts and slowly tighten, mine seemed to self bleed as I installed the cap slowly and let the fluid leak around the threads before tightening with a wrench. You can also get air out of the valve by bleeding the entire system as usual.
Disclaimer: Mess with this valve at your own risk!
I'm thinking weak front brakes on your car are causing the rears to do more work and locking up on gravel is a symptom of this.
All good sugestions by Robert Kane, I would investigate those as well.
Hope my rambling reply helps.