While I was developing the idea for Games with Toys, I tried to find toys, since the ones from my childhood were gone. Here’s what I found: it really is possible to find toys at very low prices.
I went to several stores. They clearly weren’t the best option; far from it, since prices are higher than elsewhere. Besides, toy soldiers are hard to find in those stores.
This being said, dinosaurs are easy to find! Inexpensive bags are perfectly adequate, or if you feel like treating yourself, you can buy high-quality products like Schleich, which are simply sublime.
In the toy section of most supermarkets, you will easily find bags of inexpensive toy soldiers and dinosaurs. While you get your groceries, you can pick up adequate miniatures. They may not be the best-looking ones, but at these prices it would be a shame not to buy them.
Flea markets and secondhand stores
With a bit of luck, you will find lots of toys that are practically free (like the velociraptor that inspired Dino Soldiers). You might also find pieces to set the stage, like palm trees, skeletons or decorative rocks for aquariums.
This is the obvious source. And supply is plentiful. Don’t overlook bargain websites, where you can find treasures, often in batches. But be wary of delivery delays. A lot of them are imported from China and you will need to wait a few weeks.
I have a soft spot for the soldiers of the Tim Mee brand, which are just perfect. (Watch out for the very high import costs towards Europe.)
There are a staggering number of vendors, and I am not the best person to advise you. Please share your own sources on my Discord.
Remember, it is very easy to find the material you need, in your price range. The most important thing is to have fun.