Many caterpillar species hide in the foliage of their foodplant while they are feeding and growing, but go walkabout when they are mature and need to pupate. So if a caterpillar is found walking about rather than on its foodplant, it probably does not need any more food.
There are two exceptions to this. If you notice a totally defoliated nearby plant, bush, or tree, in this case it may be seeking more sustenance. Or if was found on an old fence or wall, it may feed not on leaves, but on Algae and Lichens. So choosing a diagonal stick with algae and/or lichen on it may solve this possibility.
If the caterpillar is immature and needs food, the easy choice of what to feed it is leaves of the plant species it was found on. Most caterpillars are fussy eaters. They can tell from the smell, taste and texture, whether a leaf is good for them to eat or not. Most caterpillars will just refuse to eat anything that they think is poisonous to them.
Over the last 100 million years : there has been a silent but escalating war and arms race : between plants and caterpillars. Various plants have evolved into over 100 different plant families, each producing different poisons to stop caterpillars eating them. Meanwhile the caterpillars have co-evolved into over 100 different families, each developing different metabolisms, each equipped to cope with the poison typically from one plant family. So incidentally, if you can find the name and family of the caterpillar's foodplant, then that will help identify the caterpillar.
The five basic common Australian caterpillar foods are:
If an unknown caterpillar rejects all five, then rose petals and thinly sliced apple are good standbys. Apparently plants in Rosaceae have developed very few poisons.
(written 11 March 2020)