Turns out I was pretty close.
I thought it might be centered around the master cylinder since the brake assist motor will run if you press the brake pedal, even with the engine off.
That's when I noticed the large solenoid control module next to the MC, and the pressure sensor coming from the side.
For some reason, I decided to squeeze the top and bottom of the control module and the brake assist shut off.
I gently pried it apart just a bit, and the motor came on again.
It all made sense. If the MC sensor detects low pressure, it signals for brakes assist - exactly the same as if one of the contacts is not making properly in the control module.
So I pulled it apart, cleaned the contacts and tightened the 2 parts with a cable tie and the problem is gone.