If you buy milk in those half gallon or gallon jugs with handles, you can use those by filling them with water, sand, cement, or dirt. The handles allow you to lift weights just as you would your dumbbells. The total weight of each milk jug will be determined by the material used to fill it. If you don't have milk bottles around, you could use laundry soap bottles with the handles filled with soap, water, or sand. However, if you don't want anything that large or heavy, you could use two small water bottles filled with water or sand.
On the other hand, you can grab a pair of canned foods to use. Canned foods, especially the smaller 16 ounce ones are really great if you are just beginning but you can use larger cans such as the #10 ones like you would use a medicine ball. These are especially good for tricep kickbacks.
You could also grab a couple bags of beans or rice and use them for bicep curls or other move needing a lighter weight. Both rice comes in bags ranging from 1 to 25 pounds but those that weigh 5 pounds or less. One can also use packages of flour since they come in 2, 5, and 10 pounds but you'd want to place these in plastic bags or shopping bags so there are handles to use. On the other hand, if you have a couple cans of paint around the house, you can use those for bicep curls or to work triceps.
Do you every buy potatoes, onions, apples, or oranges in bags? You can use those bags of fruits and vegetables as weights. You would hold them with your hands wrapped around the top. In addition, these usually come in 5 to 10 pounds and are good to replace heavier weights.
If you have 32 ounce boxes of plant based milk, soup, or other liquid, these are perfect to use instead of the normal 2 pound weights. I usually have boxes of almond milk or regular milk around the house and since they are boxes, they are fairly easy to grasp.
Speaking of shopping bags, you can put anything such as multiple cans, boxes of milk or soup, rice, beans, flour, or sugar to make them easier to use as weights. You could even use books, or logs in the bags. The use of shopping bags opens up the possibility of using so many different items. Let me know what you think, I'd love to hear. Have a great day.