When I was a kid, my aunt used to take us to "the teacher's store," a store which sold little, cheap, flimsy toys like those you'd get in a party bag in bulk for cheap.
My mom bought my son one of those small ball pits
When he outgrew it, we used to have ball wars and whip them at each other (You'd have to because they were so lightweight. Also, the plastic was thin so it would never hurt.) or gave them to teh cat to play with.