Expert Insight: What Design of Experiments (DoE) can do for you

As proud official partners with JMP software for the likes of training, consultancy, software customisation and clinical trials, we run a number of free Design of Experiments (DoE) events throughout the year pitched at professionals in research and engineering who may be new to experimental design and its numerous benefits. Following our most recent event, we sat down with JMP's Dr Phil Kay and our own Managing Director Andrew Macpherson to learn a little more about DoE, it's importance in R&D, and what support is available to those keen to begin implementing it within their company.

Why should I embrace Design of Experiments?

Phil – I think there are three levels of benefit to using Design of Experiments which I call the “Three P’s”. Predictability – it helps you get better products to market faster; Productivity – science and engineering isn’t cheap, so DoE helps you ensure you make the best use of your time and resource; Promotion – knowing DoE is a really great way to advance your career as it makes you a better scientist.

It is also worth thinking about “why do we experiment at all?”. I think non-scientists often find this a bit confusing: “Why do you need to experiment? You learn how biology, chemistry, physics works at university, so you have got the answer to everything, right?”. Well, industries like pharmaceuticals, biotech, material sciences, semiconductors, consumer products and foods are characterised by having tricky process technology. It is why certain companies are successful and have a competitive advantage; their technology is difficult to do, and you can't just copy it. However, the other thing that characterises these industries is the sheer number of different things that can be affecting their processes. The only way we're going to understand how any number of factors affect a process, and the overall responses, is by actually experimenting… by changing things… and this can get very complex and costly! We need to be smart about how we do this! This is what design experiments is all about; we are ultimately trying to understand our processes.

I want to start taking a DoE approach to my work… but where do I start?

Andrew – The successful implementation of Design of Experiments will always revolve around the science; it is always going to be at its most successful if you can feed the right information in to begin with. As such, the selection of the factors that you want to explore, and their ranges, is not a trivial exercise – without doubt, that is the trickiest bit. But, if you can define your inputs and your outputs, then DoE will enable you to effectively and efficiently optimise your process, assay, formulation or product across as many outputs as you wish!

How do software packages like JMP help me design an experiment and then analyse the results?

Phil – For me, when I first saw JMP’s prediction profiler it was something of an epiphany. I was working as a process development chemist developing toner for laser printers. Throughout my education I thought my job as a development chemist was to make “stuff” and make that “stuff” better. However, my job was actually to gain understanding about the processes. It wasn't about the stuff that I made - which would go in the bin - it was about how we made this stuff at the 2000 litre scale, how we can get it to our customers around the world, and how we can do that cost-effectively, and ensure consistent quality.

Setting up a design and analysing it in JMP helps us achieve this, and the prediction profiler is a particularly useful tool. For example, let’s say “yield” is the important outcome - our important response – in our process. By using the profiler, we can see how – say – changing the amount of catalyst or changing the temperature might affect the yield. We might also see that time has a complex, curvilinear relationship with yield, or see that maybe “feed rate” or “agitation” have no effect. These discoveries may in turn lead to a richer understanding of what is going on; for example, we may see that at low temperatures, catalyst has a very important effect, whilst at higher temperatures, catalyst doesn't matter as much, so we don't need to add as much in. We could also find the optimum settings; we can tweak it to find settings that are robust, so it doesn't matter how we change catalyst and temperature. This is the kind of understanding we're trying to generate about our processes.

What other DoE support is out there for me?

Andrew – Whilst there are several excellent online training resources available for those who wish to self-teach, the team and I at Prism Training & Consultancy have been delivering Design of Experiments workshops to companies and individuals for over 24 years. Our most popular course is the Effective DoE Implementation workshop, which is designed to get people confidently setting up experiments and analysing their own studies by the time they leave the course. We always use industry-specific case studies, so the examples we feature will be ones you understand and recognise. We convey very complex statistical concepts in plain English, and tell you why you should care about them and what they mean to you in practical terms. We ensure that our courses are interactive to keep you engaged and excited about statistics, which, let's face it, isn’t everyone’s favourite subject!

Ultimately, the purpose of our training is to equip you to do DoE yourselves; we'd much prefer that you, the scientist, is the person who designs your studies, interprets the results and actually benefits from the learning directly, rather than having to rely on an in-house statistician or further external support. We want you to see how DoE can work for you. One recent piece of feedback we received said “DoE is impactful thing I've learned about since undergrad, more valuable than a PhD”, which speaks volumes for the power of these tools.

“Plan beats no plan”.

Phil – I do like quotes about DoE. Timothy F Geithner, the Treasury Secretary under the first Obama administration when the Finance world was falling apart, was often involved in a lot of crises meetings about bank collapses. He was very fond of saying “Plan beats no plan”. Why is this relevant for DoE? There is no such thing as the “right” designed experiment – you only know that once you have actually got the results. Any plan for an experiment is better than not having one. Just doing a random One Factor A Time through the experimental possibilities is not a good way of doing it. If you don’t have a plan, you just have to rely on hope; Geithner was also fond of saying “hope is not a strategy”, and if you’re not using DoE, you’re just relying on hope!