Quantitative and qualitative analysis in website optimization

Website optimization and isolating key areas for conversion growth may cause a lot of problems. Where to start? What kinds of change should be considered? What is the most important?

To answer these and many other questions it’s good to have a closer look at quantitative and qualitative analyses and how we can use them in our daily work on websites.

Quantitative and qualitative analysis characteristics

Research methodology is a set of basic rules aimed at providing us with a significant measure from the point of view of our business objectives. Of course, these rules are being improved constantly striving to provide us with more and more faithful results and answer the questions: how to carry out research, who should be tested, in what circumstances and by what techniques or tools. Regardless of the changes taking place, quantitative and qualitative analysis can be included in the fundamental research methods.

So, let’s have a closer look at what are both methods characterized by and in what situations can they help us:

Table 1.Characteristics of qualitative and quantitative analysis

When we know the difference between a quantitative and qualitative method, let’s see how we should choose between the two methods:

Table 2. The selection process for research methods and data collection

Properly organized research plays a key role in website optimization. Knowing the differences between various methodologies can save our time and money. Organizing it and carrying out properly directly affects the received conclusions, thus also the effectiveness of subsequent decisions.

Mixed Methods Research – a synergy of quantitative and qualitative analysis

It’s worth mentioning here one clear phenomenon. The topic of quantitative and qualitative analysis synergy is currently one of the key trends in development of every e-business. Representatives of both analyses have waged a war of paradigms with each other since time immemorial, seeking to prove the superiority of one method over the other. Nowadays, more and more often these approaches complement each other forming so-called mixed methods research and providing full spectrum of answers to posed questions.

According to the definition from the quarterly Journal of Mixed Methods Research, mixed methods are: “(…) an inquiry in which a researcher collects and analyzes data, integrates the results and draws conclusions using both qualitative and quantitative approaches and methods in a single project or research program.“(1) An example of such situation may be we using qualitative analyses to form hypotheses test in a quantitative manner, or using qualitative research to interpret previously collected quantitative empirical data.

The importance of mixed methods was also stressed at this year’s UX Poland conference by Jim Sterne, the founder of Digital Analytics Association, who claimed that only by combining quantitative and qualitative analysis with rapidly evolving technology we can talk about the real effects of website optimization.

Of course, the exact scenario we’re going to use depends greatly on our current needs and goals. To better visualize the process of combining both types of information, let’s look at two examples:

Example 1. Moving from quantitative research to optimization activities

Let us assume that based on aggregated data taken from the sales system (quantitative research), we see that users are more likely to buy products from Computers category than products from Laptops category, despite the fact that products in Laptops category have more attractive price. Let me say here that both categories have a very competitive offer and are characterized by similar seasonality. To understand this phenomenon we decide to carry out in-depth interviews (qualitative research) with the main purpose of examining which elements affect shopping habits. During the research, we are able to create three sample hypotheses:

The majority of users reaches Computers category through advertising slider on the home page. Laptops category is not in any way visible or promoted there. Thus, users are not aware of the price differences;
Most of the users are business customers who, due to the nature of their business, are interested in buying only desktop computers for their offices. Therefore, they are consciously not interested in laptops;
Technical descriptions, a key factor in decision-making, are more developed in Computers category, than in Laptops. That is why the user often ends up buying the first product, despite the higher price. These hypotheses are checked through analysis and data mining (quantitative research). Reached conclusions are subject to additional verification through multivariate tests (quantitative research) that help us choose the best solution. Finally, we implement the changes on the website.

Example 2. Moving from qualitative research to optimization activities

We’re in the middle of implementing website redesign. It was based on previous in-depth interviews and tests with users (qualitative research). After launching the website we are curious how many users perform certain actions on the site and whether key functions achieve their target. Using dedicated measurement done by analytical tools we aggregate necessary data, e.g. the amount of abandoned order paths (quantitative research). The resulting information becomes the basis for carrying out open surveys with users to identify the reasons for such behavior (qualitative research) and introducing fixes to the project. The proposed conclusions are verified with A/B tests. Finally, we implement the changes on the website.

Regardless of the methodology we start from, the process of Mixed Methods Research should be performed periodically, in weekly or monthly series, using agile methods of Lean UX.

Summary

In contrast to analytical tools – which grow in numbers year by year – research methodology is an element that doesn’t undergo such rapid changes. This allows us to treat it as a signpost of planning optimization measures. At the same time, we should bear in mind that the greatest impact to improve the website is obtained using Mixed Methods Research approach in our daily work.