A couple questions for the OP. Is your caddy the newer version that has the digital touch screen that controls fan speed based on plenum temp? Is the Caddy correctly sized to work with the sq/ft of your home? I have the larger Max Caddy version. Between my home's size, the amount of very tall windows I have, and the extensive duct network the heat needs to pass through to get to the second floor and room over the garage, my unit can barely keep up when it gets below 10F and the winds pick up for any significant amount of time. This is with wood under 10% MC and draft set to .05 too. My intake damper will stay open almost the whole time when temps are this low and despite what others above have said, the plenum temp on mine goes WAY down when the intake damper closes. That said, the lack of secondaries in your burn is different than mine. I get a raging fire from the wood load and also get nice secondaries rolling from the air tubes when the intake damper is open.
What I do agree with from the above posts is that moisture content and draft are likely a big part of the issue. To start, spend $30 on a moisture meter from Amazon. This one is popular:
General Moisture Meter Then you need a manometer for your stove pipe to check draft. Most folks like the Dwyer liquid filled but I preferred this style. It's more expensive but a great gauge:
Magnehelic
To use the moisture meter, you need to do a fresh split on a piece from your supply and insert the probes vertically or with the grain. Report back on your findings and I bet that will be the issue with your moisture content. To install the manometer, you simply need to drill a small hole in your stovepipe after it comes out of the furnace's collar. Insert a piece of 1/4" metal tubing in the hole and point it downstream. Once away from the heat, adapt that metal tubing to rubber 1/4" and connect it to the gauge. If you are over drafting, you'll need a barometric damper on a vertical section of your stovepipe. It needs to be as level as possible. Here is an example:
Field Damper Once installed it will be a very simple process to dial back the draft. You simply loosen the adjustment knob and slide the weight forward until you get to between .04 and .06 on your gauge and tighten it back up. (If your wood is damp, keep it at .06 for this season to help get your temps up.)
Now, back to my first original question. If you have the digital control / variable blower speed and your wood is damp, your plenum temp will rarely get high enough to reach the higher blower speeds. Depending on your ductwork volume, if you're stuck on speed 1, you're getting hardly any heat at the end of your duct run. On my Max Caddy, I need to see a sustained plenum temp (shown on the home screen on the digital control) of over 140 to kick the blower speed up. If it's not moisture content keeping your plenum temp down and your blower speed low, it could be static pressure. My house has three zones. On very cold mornings when all 3 zones are calling for heat, static pressure is too low with the amount of air going through the furnace and no matter how hot the fire gets, I'll never get the plenum temp to rise enough to get me to the next blower speed. What I've found I have to do is shut off the first floor zone (which needs the least heat if I leave the cellar door open and allow the radiant heat to rise). Shutting off zone 1 will increase the static pressure enough to get the plenum temp up enough to get me to speed 6 and we are off to the races with the house recovering to comfortable temps.
One last question for you is how well are your ducts insulated? Are they all inside the conditioned house envelope? Mine for the second floor and room over the garage were in unconditioned space and while they were insulated by the HVAC installer, they needed more. Sprayfoam made a world of difference in retaining the heat in the ducts and keeping the air in the returns from dropping further.
You may not have all these issues. However, I know how each contributes because when I built my home in 2019 I did have each of these issues. Installing the manometer and baro damper were the quick easy thing to do right off the bat. Figuring out how to deal with static pressure on a zoned system took some time / trial and error. Lastly, by the following year, my wood was dry so the MC issue was gone. I still loose a few degrees when I get sustained single degree days/nights and wind but it's nothing like our first year here where it was a battle to stay comfortable. I hope these ideas help you narrow it down in your home. Report back what you've tried and how it worked out...