Potatoes are rather high in carbohydrates. They are not as bad as sugar or alcohol, but don't really fit into a low carb diet.
Low carb vegetables like cauliflower, broccoli, spinach, kale, etc.would be much better. It is up to him. The strict caveman diet is just lean meat, eggs,( and perhaps low mercury fish like sardines, herring, and salmon, although some here would say to eliminate all fish at least temporarily), and salad and steamed vegetables. I don't recommend this for a very long time, but perhaps 4-7 days just to help detoxify his body. The use of spices will also help with detoxification.

Nuts and seeds are a good source of minerals. The most important thing is to eliminate alcohol and processed sugar. I have given up fruit as well, but some people can tolerate some of the less sweet fruits.