Since you don't have apparent moisture issues in the basement, that takes care of your first step - making sure leakage isn't a problem. As for that sheetrocked wall in front of the foundation wall with the pipe along it, if it's well sealed against air currents leaking around it, then a couple coats of latex paint provide a decent vapor retarder layer to keep most of summer air moisture off the (likely) cooler foundation wall. However, much of the year there could be ground moisture migrating inward through the wall or upward from the footing, creating mold-favorable conditions behind the wall.
You spoke of later insulating under the floor above, yet you have a dehumidifier running in the basement in summer (likely a good thing), and you run the stove in winter. The other approach would be to forgo insulating that floor and just insulate the basement walls. You might think of getting behind that wall in front of the sewer pipe and applying rigid foam board above and below it as best as you can. You can seal the gap between pipe and foam with can foam. Then there shouldn't be any problem having that framed wall in front of the pipe, as long as there is no polyethylene sheet sandwiched in there behind the sheetrock. Allow for access to any cleanouts in the sewer pipe. Having part of the wall built in a way that it is easily removed for access is one thought.
When you do get into redoing the ceiling of the basement, the right way to insulate the bays at the perimeter is with closed-cell foam, either sprayed on or with "cut & cobble" application of rigid foam board sections with the perimeters sealed with canned foam. Additional batt insulation could be applied over that. As things stand (per your description), having porous insulation not totally sealed at the edges allows moisture from the air to migrate easily to the rim board, which will be cold in the winter.
Those are my thoughts, based solely on what you have described. There could be other aspects to it that would call for doing some of what I suggest somewhat differently.