The TEGs the kickstarter is using are pretty cheap I think. I see something like them for under $6 on eBay. The quality though is questionable. They could be Asian factory rejects for all I know.
Ebay and Amazon sell Peltier coolers for less than $3 each in 5-10 packs. I think I paid $5 for a single piece to tinker with.
These also generate power, but as I understand it, are optimized for cooling and not as efficient as those optimized for power production. I know you can get at least a Watt or two out of them, and probably more.
TEG Power has some optimized for power rated at up to 22 Watts for $60. Much more expensive, but also more capable.
TEG's are not only expensive, they are very inefficient (< 7%).
As solar panels show, low efficiency doesn't necessarily matter if the purchase cost is low enough and you have access to energy being wasted.
I believe one of the main cost drivers is the fact that nobody makes them in any truly significant volume. The materials are not exactly conventional, but I don't think as difficult to produce as solar cells. 20 years ago, solar cells cost $7-8 per Watt (and complete panels about twice that). In 15 years, the price dropped to just over $1 per Watt (and is now down to about $0.50/W). There were only minor advances in the technology at the time. The main thing that happened was increasing economies of scale as production grew. In those 15 years, production volume increased 40 times over.
The TEG Power thermoelectric chip above costs $3 per Watt. That's an encouraging starting point, even if the cost reductions are not as significant as what solar enjoyed.
Still, you have to find that heat being wasted and make use of it. Stealing some of the heat from your stove is a possibility. The TEG Power chip's datasheet suggests it's 5% efficient under ideal conditions. A 75,000 BTU/hour stove could hypothetically produce nearly a kW. I expect more realistically due to being limited in where you could place thermocouples on a stove and the challenge of keeping them cool, you could get a couple hundred Watts in the best circumstances.
But that would be enough to run lights and a few other devices in an off-grid cabin in a climate where solar doesn't work well.
I'm tinkering with the idea of powering a stove blower when the power is out, so I can still get meaningful heat from my insert. Energy savings beyond that if I can come up with something that will work full time would be a bonus.