One requires a permit, finishing the basement WHICH unfortunately puts it into a sort of house addition situation. Normally you can add an outlet to your house, do your own plumbing, (not alter structure without a permit) but if it's part of your finished basement plan you need an electrical permit, plumbing permit, and potentially a gas permit as well as the finished basement permit and expect 4 visits by an inspector. The furnace/water heater replacement issue, not something to be done by a homeowner. You might get away if it's electric but your heater is sophisticated. You need to calculate what water pressure as the pressure may be different than your incoming line and can be based on your highest point in your house your heating pipes reach in relation to where your heater is situated. You also may need expansion tanks, bleeder valves, back flow valves, check valves, shut off valves, flow restriction valves, pressure reducers, list goes on & on. With expansion tanks you'd need know how much to charge them, and know how to bleed your systems and purge. It's dangerous to put too much pressure into your heating system. I'm pretty sure you also can't connect it to oil/gas/kerosene yourself and lastly need to realize that just because all the parts of something is made of all U.L. parts individually, together that doesn't mean it's U.L. listed and safe. You also need to know reactions of two metals together like if you want to attach black steel to copper you make a weak battery in presense of fluid and one of the materials will rapidly decay. A die-electric union needs to be used between the two, things like a mixing valve needs to be piped at least 12" below the hot water out of your heater, and the expansion tank positioned at least 18" away from the vertical run off your heater. The list goes on, and on, and on.
For finishing basements, the first step is the Rough-In. Where, they do things like check that your ceiling is no less than 7' and any habitable room no less than 70 square feet. You need to bring them your plans. There are certain allowances for ceiling heights less than that, say for duct work they allow the ceiling to reduce to 6'6" for it. Foundation needs to be insulated to minimal levels in your plan, and they'll want to know if you're adding bedrooms in the basement they MUST have a window and will want to know what size. It must be used for egress that usually when opened, the opening must be at least 20" wide and open 24" tall and not need anything special to open (no crank, etc). Usually any habitated rooms need 8% of window space compared to floor area. They'll want AC powered U.L. smoke detectors in vicinity of bedroom entrance, inside any bedrooms, and at the bottom of any stairs leading up and interconnected. Clothes dryers last I checked are one of the leading causes of house fires. The must exhaust to the outside and use only metal pipe. Flex metal pipe only allowed directly off the dryer to connect it with the solid sheet metal pipe. No screws are allowed to penetrate any connections in this pipe only clamps and tape to fasten joints. All bathrooms usually must include a 20amp GFCI outlet on its own breaker. You need a wall mounted light switch in every habitable room, bathroom, hallway, stairway, and exterior doors. The one near the stairway needs to be illuminated. You also need to make sure you have at least 1 light in each storage room, or area containing heating/ac equipment.
Okay, to cut this short with the finished basement after the application and the "rough-in" appointment (before any work is done)
They come back for the "Building Framing" inspection where they inspect all the electrical, plumbing, and gas work is done but before covering is applied to the walls or insulation.
They then come back for the "Building insulation inspection" where they check that you properly insulated.
The last is Final inspection.
Anyway, that's usually how a finished basement goes. The kicker is usually the ceiling height, you're required to have 7' ceilings last I checked. I'm not an expert on the topic, but have seen a few basement finishing models being done. My basement is finished, and has a permit. Also, there may be some strange laws. For example, I can NOT have a kitchen in my basement. What's considered a kitchen? An oven. I can have a fridge, stove top, microwave, but can't have an oven. Putting one in, sets the flags off my house is a two-family and it's not zoned for it. So, I can have everything in my finished basement except an oven which... is exactly what I have (kitchen sink, fridge, faucets, full bathroom, oven top, microwave... no oven so can't cook turkeys down there).