The biggest obstacle will probably be avoiding changing more than one part of the process or ingredient at a time through the whole project. The biggest detriment to the experiment will be if you change what recipe you're making, unless of course, you're just trying the change for a particular beer style/recipe. That means a LOTTA brewing and a lotta tasting (darn the luck), spanning a long time, which could affect the outcome of the beer because of aging. Aged beer is better. Full stop. Even the stuff I don't particularly like is better after aging. The fun part will also be keeping a good journal and log of what happens on BOTH batches running in tandem. Any variation on either mini-batch will basically mean they aren't the same batch any more.
Up to a point, you may find you can do some large batches and split them for your testing, but not if the test parameters are anything in mash or boil time. Once you get past boil stage, though, you'll have the best consistency by splitting a single large batch. Depending on your hops and bittering targets, longer boils are certainly going to raise the bitterness and hurt some of the hop taste. For that part, I'd probably target the most neutral hop possible with ONLY bittering qualities for the target to avoid cooking off the flavors. Unless, of course, that's one of the test targets.
It might also get a little boring drinking the same brew over and over by the time you get all the tests run. That may jade your opinion a little as the project wears on. Make sure to keep something around to keep the taste buds happy and in tune, as well as at least one prior iteration of the testing so you can side-by-side the results from two different parameter changes. You may find a degradation that would otherwise go unnoticed.
Lastly, it's going to take a lot of work to control all variables except the one you're testing, and face it, sh** happens. Murphy's 1st law: Don't mess with Mrs. Murphy. 2nd Law: If it CAN happen, it WILL happen and at the least opportune time.
But in absolute support of your efforts, this is exactly some of the things I want to try as well. Tweaking the processes and procedures with the primitive equipment I have may help me down the line to keep improving my brewing and get a lot easier when I have more modern hardware. Just because something "has always been done that way" doesn't mean we can't find improvements or at least figure out WHY it's always been done that way. The real trick will be to get the same opinion of the change from someone else. If you're at this next winter (summer to me), I'll be more than happy to come by and help you with the tasting if we can make our planned trek to Oz. Tentative dates at the moment are mid-June to mid-July. I've grown fond of celebrating my birthday in other countries. I think we'll do a full month this time. Normally, it's 3 weeks because we normally have to get someone to look after kids while we're gone. Not gonna be an issue next time. Even if I have to buy someone a tent.
Gonna have a look at those 8L PET mini-kegs. I like what I see there, and should be able to fit a few of those in the freezer, expecially standing up. That would allow brewing several batches at shorter intervals.
Is your plan to test something, decide Method A is better than Method B, and then only do the rest of the project with Method A? It may well be that the next variable you change could drastically affect the previous outcome, or vice versa.
