So I was driving my Previa tonight and I noticed that the brakes weren't being consistent in how much/hard I have to push down each time. Some stops the brake pedal would be firm and go halfway down (normal) and I would stop fine. Other times the brake pedal is really soft and I find myself putting my foot to the bottom of the floor in order to get the car to stop. The latter of the 2 really scares me when driving and wanted to know how to fix the problem. I was thinking that maybe the brake fluid level was low, but that would't result in inconsistent brake pedal, is would just be consistently at the bottom.

Any other suggestions to solve the problem?