Smoke

when this happened to us it was because of a negative pressure situation and we had to warm the chimney before use. Basically with a hair dryer for 5 minutes till we felt no cold air coming down the chimney. You can also light paper and that warms it very quickly. If we let our stove go a couple of days between fires we have a smoke issue if we don't reverse the flow first.

Does that help?

When did you last have the chimney swept and/or inspected? Is there a cap on the chimney with a spark arrestor? If so, can you see from the ground with binoculars if the screen/mesh is clogged with creosote? Whenever you get smoke coming out of a fireplace, stove, or insert this is always a good place to start looking for the cause since it is easy to spot and easy to remedy.

Post pics of fireplace and outside chimney