Fan curve help

I'm setting up a fan curve in my air 540 case and I have an h100i pro RGB cooler in push/pull configuration with four 1500 RPM SP fans. I have positive airflow into the case (not sure if that is relevant) with two 140MM 1700 RPM fans in the front pulling air in and the 140 fan blowing out of the back.

Should I set two fan curves, one for the case fans and the others for the AIO using the coolant temp sensor for both? And what should the fan curve be set at ideally?

The balanced curve I am using is producing higher temps than I would like with Prime95 stress test. I'm getting temps in the 80-90c range with occasional spikes to 95c before it falls around 83c.

Your fan speed on the H100i will only affect the coolant temp, listed as H100i Temp in Link. For something like Prime 95 or any other stress test, the cooler is not the limiting factor. The limitation is the CPU temperature on the pin side of the chip, where the heat is generated. There is nothing you can do about that and all modern CPUs are limited by voltage and CPU thermal properties. Your change in coolant temp is likely only 6-8C and thus the most you could reduce temperatures with any radiator or fan combination is only 2-3C. Also, there is no reason to tune your system up for running Prime95, unless hunting Mersenne Primes is your normal hobby. If you are going to run any stress test, the best thing to do is set a static fan speed for the duration of the test. This makes it easier to compare results between instances are eliminates the fan curve as a variable.

As for what to do with your 3 case fans, that depends on what control options are available. Motherboard options may not be great on the Z270. CPU temp is about worthless. I do not think VRM temp was an option on that series. The best bet would if you have a single thermistor slot on the board and the temp probe end could go above the GPU somewhere. That would create a large enough data range to make subtle fan changes and in response the largest heat producer in the case. If you have other control options, you'll have to elaborate.

You will have little trouble creating positive pressure in a 540. Your front fans will move quite a bit of air and the top 2x120 on the radiator move very little because of the resistance from the fins. At maximum speed, they will only move about half their listed CFM - even in push-pull. Regardless, you can always alter pressure balance with changes in fan speed and a slight front bias should ensure that.

I'm running everything connected to the commander pro to control the fans speeds. As for the radiator I have four fans total pulling air into the case. They are SP noctua fans so they seem to work pretty well to keep the dust out. the case fans in the front are SP fans as well as they are pulling through the filter on the front. I have more fans laying around I could add if I really needed to. I believe I have 2 more 140 SP fans and the original case fans that come with the air 540. Seems redundant to have more than 7 fans total, but I may be wrong as I want static pressure to keep the dust out.

on the curve side I did read the water cooler guide (very helpful) and I am setting a custom curve with the coolant temp as the sensor or all the fans in the case at the moment. my ambient temp while web surfing and idle is at 27.5 C. and in the guide it says that at max load it should hit around +6 C or so. If that is correct I should be at max fan speed at 33.5 C if I am understanding correctly. then I just need to set the ramp up for load (sometimes I video encode) to keep The CPU under 80C under heavy load if I can. I was concerned when it hit 95C before I started tweaking the settings.

I only use prime95 to stress test for temperature purposes, not doing any kind of crazy OC as I'm still running a 7600k running TUPII in bios. I believe it set it to around a 26% OC so nothing crazy.

edit:So I ran the intel utility this time and I set the AIO fan curve to a reasonable curve and I don't know what it was but that seems to have made a huge difference. The water temp never went over 31C and the CPU stayed under 73C. I can't set the curve on the pump speed itself but when you hover over the icon to the right of the variable temperature to monitor (Coolant temp) it says that setting the fan curve will affect the pump setting globally. So, I'm guessing I inadvertantly had it set to quiet mode and just didn't change it.