Heat oil or butter in large nonstick skillet or electric griddle over medium high heat.
In a large bowl, whisk together flours, baking powder, salt, sugar, cinnamon, nutmeg and ginger.
Add eggs, sweet potato, milk and vanilla extract to flour mixture. Whisk everything to combine until batter is mostly smooth. Add more milk as needed to get a nice pancake batter consistency. A few lumps remaining is totally fine.
Spoon even ladle-fulls of batter into skillet or onto griddle. You can also use a ¼ measuring cup to ensure you get even sized pancakes.
Cook until bubbles for on top and pancakes are golden on the bottom, 2-3 minutes, then flip. Continue to cook until other side is golden brown and pancakes are fluffy, 2-3 minutes more. Serve pancakes warm with whipped cream, cinnamon and maple syrup if desired!
If you do not have coconut flour, feel free to use all whole wheat or all purpose flour. You will need a little less milk if not using the coconut flour.