Smoke smell from fireplace

[Jedi]

So I am getting smoke smell from the fireplace on these humid rainy days in Maryland. Once the weather outside gets hotter, it seems to create a pressure difference and air from the chimney is somehow leaking inside the house. To clarify, I have a wood insert installed with a liner. The liner doesnt have any leaks. I can smell the smoke air from the shroud sides, so it's the space between the chimney and the liner. If I open the insert door, it smells 'fresh' inside since the insert was already swept. Are there any remedies to fix this, or what can be causing it? I don't suspect a loose cap at the top, but could it be this? It's annoying because the whole room smells smokey.

In the picture, the red arrows and blue dots show where the smell comes from.

 

[EbS-P]

Was the fireplace and its flue well cleaned before the insert was installed? Do you have well seal and insulated blockoff plate? Smell is probably not the new instep but the old fireplace and or flue.

 

[Woody Stover]

Yep, the "stack effect" of the house is probably responsible. Hot air rises and escapes the house upstairs, drawing in air to replace it from wherever it can, in this case your smelly chimney. EbS-P has covered the possible solutions.