Methodology

 

MECLABS is a science lab (primary research) with a consultancy (applied research). We built the first research program to conduct rigorous experiments in the new science of optimization. Our discoveries help leaders optimize the financial performance of their sales and marketing programs.

MECLABS gains this knowledge by utilizing the scientific method to create real-world experiments and then extracting successful principles based on the data generated by this experimentation.

To ensure a consistently high level of scientific rigor in this experimentation, MECLABS’ research analysts utilize a uniform test protocol and series of heuristics to help analyze real-world data and create new single- and multi-factor tests that can reveal further discoveries.

Three Defining Characteristics of the MECLABS Methodology


1. Scientific inquiry

Our research is conducted according to the principles and practices of experimental science. Experiments are designed and conducted in accordance with a larger research plan using a formal test protocol, developed by MECLABS exclusively for its research. Research questions are formed and evaluated as hypotheses, which are tested and the outcomes are statistically validated.

Our research methodology allows us to make the most of any of the widely implemented testing and analytics solutions currently in use (such as Google Website Optimizer, Omniture Test&Target, Google Analytics, Omniture SiteCatalyst, ExactTarget, Eloqua), whether independently or integrated within our own testing platform (the MarketingExperiments Lab™). Using these testing and analytics solutions, we conduct single-factorial (A/B), multi-factorial (multi-variable/MV) and sequential (if unavoidable) tests.

Experimental subjects are most typically pulled from a continuous, real-valued random sample population of consumers, and occasional focus group testing is performed, especially in new product development situations. The full spectrum of marketing communications is tested, from web pages, email messages, banner and text ads to traditional channels like TV, radio, and print media that drive traffic to an online platform (such as the desktop or mobile Web).

2. Real-world experimentation

MECLABS empirical research focuses on real-world Internet marketing challenges and opportunities. By conducting experiments with actual commercial offerings, we uncover relevant, practical insights that enable our Research Partners to discover what works – for their products, with their target customers, in today’s ever-changing marketplace.

The expertise of MECLABS' scientists and analysts is informed by the hundreds of experiments they conduct and participate in each year. Their experiments are informed by these insights, combined with the discipline and wisdom that come from both their formal training in the science of Cognitive Response Optimization and the experience of their prior research.

As technology and culture evolve, MECLABS continues to stay on top of what works best by selecting and collaborating with a small set of organizations capable of responding to and benefiting from increased demand for their products – those with the infrastructure and executional capabilities (e.g., inventory or fulfillment system) that can handle a significant increase in demand.

The findings from these real-world experiments are added to the MECLABS research repository – which we believe to be the most extensive body of experimental research data related to online marketing and made available at no charge to professional marketers through articles, case studies, research journals and webinars.

3. Data-driven principles

MECLABS is a laboratory that uses empirical, relevant data to craft and teach Internet marketing principles. By conducting experiments instead of just relying on so-called “marketer’s intuition,” we are able to discover what really works today. MECLABS' approach transcends “best practices” by discovering and applying principles that help determine when each widely accepted “best practice” works (and when it does not).

The goal of these studies is to discover the most effective way to get prospects to take the desired action, whether it's purchasing a product, starting a subscription, completing a lead form, making a donation, or initiating some other action. We translate the insights garnered from helping actual companies reach tangible objectives into teachable principles that can benefit our worldwide community of over 200,000 marketers.

Through more than ten years of extensive experimentation and study, MECLABS research analysts, led by Dr. Flint McGlaughlin, have replaced literally hundreds of special case practices with a unified set of core principles defining and describing the Offer/Response-Optimization process.

These principles are so well defined that they have been taught to hundreds of professional marketers through the MECLABS Training and Certification Program.

The MECLABS Approach to Experimentation

Experimental science consists of a repeated process of observational inquiry, hypothesis formation, testing, analysis and interpretation. MECLABS has developed and applies a decidedly practical form of this discipline which is centered on an empirically derived model of human behavior, motivation, and decision making consisting of a small set of core principles, or “heuristics.” This model provides proven principles for the planning, design, conduct and interpretation of conversion optimization tests, thereby greatly reducing the time-to-discovery – and to increased demand.

1. Heuristics

Several key elements of the MECLABS methodology are formulated using a heuristic approach. That is, tools to help organize one’s thinking and analysis of a given product offering, marketing message, or experimental design.

For instance, our Conversion Index analysis is based on the well-known Conversion Sequence heuristic. This empirically derived framework brings structure and clarity to analysis of the conversion process and directs the focus and prioritization for optimization energy. The Conversion Index is expressed as follows:

C = 4m + 3v + 2(i-f) - 2a ©

Wherein:
C = Probability of conversion
m = Motivation of user (when)
v = Clarity of the value proposition (why)
i = Incentive to take action
f = Friction elements of process
a = Anxiety about entering information

Please note: For the purposes of the MECLABS testing methodology, Friction is defined as a psychological resistance to a given element in the sales or sign-up process. Anxiety is a psychological concern stimulated by a given element in the sales or sign-up process.
Another popular heuristic is the Email Messaging Optimization Index:

eme = rv (of + i) – (f + a) ©

Wherein:
eme = email messaging effectiveness index
rv = relevance to the consumer
of = offer value (why)
i = incentive to take action
f = friction elements of process
a = anxiety about entering information

These are only two of the heuristics we deploy on behalf of our clients. They are only examples of the codified approach we have developed to facilitate a consistent, repeatable approach to testing. The application of the above heuristics, as well as the others quantified in the MECLABS research process, allows us to empirically extend these findings to any number of business partners.

2. Data analysis

MECLABS testing methodology relies extensively on data analysis in three key areas: understanding target customers, identifying optimization opportunities, and assessing test outcomes.

Creating the optimal message and process for the target customer is the essence of optimization. MECLABS meticulously studies all available historical data, from web analytics to transactional and demographic records, to understand existing customer segments. Through thousands of projects, we have discovered that identifying these segments early allows us to appeal to each in the most effective way, resulting in greater improvements than those afforded by optimizing in aggregate. In multi-product scenarios, segmentation analyses have helped us determine the right offer, the right up-sell and cross-sells, and the right prices for each segment.

Today’s Web marketer is faced with a perpetual parade of competing metrics software and testing tools – each with its own strengths and limitations, owing to its particular approach and nomenclature. MECLABS’ approach to web marketing optimization is science-centric, and consequently can be robustly applied using any commercial-grade web metrics or testing solution – even across different systems.

While all of the current commercial grade testing and metrics systems help you with how to test, the greatest obstacle to breakthrough optimization gains is establishing what to test. The result is, marketers get seemingly significant results, masking the fact that testing a different page, or a different step in a process, would have produced something far more meaningful and profitable.

3. Single-factor and multi-factor testing

After analyzing the Internet marketing challenge using applicable heuristics and analyzing historical data using various established statistical tools and predictive algorithms, MECLABS research analysts develop a hypothesis (or multiple hypotheses) and design real-world experiments to test them. According to the testing objectives, as well as the available tools and traffic, experiments are designed to maximize the value of each test to achieve the greatest long-term value in the shortest period of time. Typically, that includes a mix of single-factor (a.k.a., A/B) and multi-factor (a.k.a., multivariable or MV) tests.

4. Test protocol

The Test Protocol is a software-based tool developed by MECLABS to guide and facilitate the experimental testing process. It is the document of record for each test, and the repository of Test Protocols is an essential component of the MECLABS knowledge base.

The design of the Test Protocol is continually refined through ongoing experimentation. The composition, structure, and sequence of the Test Protocol are designed to guide the process and enforce the standards and discipline of scientific experimentation aimed at hypothesis testing.

Depending on the key metric, a statistical test planning tool is then used to estimate the projected sample size required and to project test duration.

As the test data is gathered, it is accumulated in the test protocol and the computations refreshed periodically to test for validity. Depending upon the experiment design, a single test sequence may involve an iterative approach to identifying and pruning out under-performing treatments while adaptively introducing new variations as informed by the prior recursions. This approach can be used to accelerate the pace of discovery with minimal risk of missing an optimal solution.

Once the optimal experimental treatment is identified – or the design sample size is exceeded and the test is deemed inconclusive (i.e., based on the data received, a winner cannot be chosen in an acceptable time frame) – the complete data set is analyzed in the context of the original set of hypotheses to evaluate whether they are supported by the experimental data or whether they must be rejected. While being right is usually gratifying, we often learn more from the tests that prove us wrong than those that suggest that we had it right – for therein lie the seeds of new discovery.

The last step of every test sequence is interpretation and recommendations for further research. Since the test hypotheses, and the treatment designs, are informed by the then-current body of MECLABS optimization principles, this step ensures that MECLABS continually stays on top of changes in “what really works” as markets, tools and cultures emerge and evolve; and the principles thereby remain “evergreen.”

MECLABS data analysis capabilities and methods are at once regulated by and serve to enable the experimental research conducted by the Applied Research Division’s Conversion Group.


Applying this Methodology across all MECLABS Groups

All of the MECLABS Groups - Strategy, Conversion, Leads, Agency, Technology, and Training - leverage this methodology as a core principal of our value proposition. The consistent application of this process allows each Group to continually codify new findings and develop new insights. These insights are then applied to the practices of all other Groups.

The Conversion Group deploys testing and optimization techniques to our partners' web sites and email campaigns. As we continue to discover what really works in the sales and marketing process, those findings are formalized and disseminated in both our Primary Research groups and our Training Group. The application of necessary technologies in this process leads us to deploy a continual stream of best practices in the Technology Group. Our constant focus on the deployment and management of marketing campaigns allows us to apply our expertise via the Agency Group. The Leads Group uses these discoveries, combined with the human touch, to serve as a bridge between your sales and marketing organizations. The leaders of these Groups form our Leadership Team and their wisdom can be harnessed en masse via the Strategy Group.