Bernoulli's Principle states that as the speed of air increases, its pressure decreases. When you blow a big breath near (but not on) the opening of the bag, you increase the speed of the air molecules and create an area of low air pressure. The surrounding high-pressure air rushes in to fill in the low-pressure area you created ("winds will blow from high to low"). As a result, a large quantity of air from the atmosphere is drawn into the bag at the same time as you blow into the bag.
If you blow with your mouth right on the opening, the only air going into the bag is the air from your lungs. You need a lot more breaths to fill the bag this way.