What Three Factors Make a Design a True Experiment

True Experimental Design

What is a True Experimental Design?

True experimental design is a statistical approach of establishing a cause and effect relationship between different variables. This is one of the most accurate forms of research designs which provides a substantial backing to support the existence of relationships.

There are three criterias that need to be met in order to carry out a true experimental research:

The existence of a control group: The sample of participants is subdivided into 2 groups – one that is subjected to the experiment and so, undergo changes and the other that does not.
The presence of an independent variable: Independent variables that influence the working of other variables must be there for the researcher to control and observe changes.

Random assignment: Participants must be randomly distributed within the groups.

See Voxco survey software in action with a Free demo.

An example of True Experimental Design

A study to observe the effects of physical exercise on productivity levels can be conducted using a true experimental design.

Suppose a group of 300 people volunteer for a study involving office workers in their 20s. These 300 participants are randomly distributed into 3 groups wherein the first group is a control group which does not participate in exercising and has to carry on with their everyday schedule. The second group is asked to indulge in home workouts for 30-45 minutes every day for one month. The third group has to workout 2 hours every day for a month. Both the groups have to take one rest day per week.

In this research, the level of physical exercise acts as an independent variable while the performance at workplace is a dependent variable that varies with the change in exercise levels.

Before initiating the research, each participant's current performance at workplace is evaluated and documented. As the study goes on, a progress report is generated for each of the 300 participants to monitor how their physical activity has impacted their workplace functioning.

At the end of two weeks, participants from the second and third group that are able to endure their current level of workout, are asked to increase their daily exercise time by half an hour while those that aren't able to endure, are suggested to either continue with the same timing or fix the timing to a level that is half an hour lower. So, for example, a participant who at the end of two weeks is not able to put up with 2 hours of workout, will now workout for 1 hour and 30 minutes for the remaining tenure of two weeks while someone who can endure the 2 hours, will now push themselves towards 2 hours and 30 minutes.

In this manner, the researcher notes the timings of each member from the two active groups for the first two weeks and the remaining two weeks after the change in timings and also monitors their corresponding performance levels at work.

The above example can be categorized as a true experiment research as:

There is a control group involved: Group 1 that carries on with their schedule without being conditioned to exercise.
There is an IV for the researcher to manipulate: the duration of exercise each day.
Random assignment: 300 participants are randomly distributed into 3 groups and as such, there is no criteria for assignment.

Purpose and usage

Both the primary usage and purpose of a true experimental research design lies in establishing meaningful relationships based on quantitative surveillance. True experiments focus on connecting the dots between two or more variables by displaying how the change in one brings about a change in another variable. It can be as simple as having enough sleep improves retention or as large scale as geographical differences affect consumer behaviour. The main idea is to ensure the presence of different sets of variables to study with some shared commonality.

Beyond this, True experimental research is used when the three criterias of random distribution, a control group and an independent variable to be manipulated by the researcher, are met.

Advantages of True Experimental Design

Concrete method of research:

The statistical nature of experimental design makes it highly credible and accurate. The data collected from the research is subjected to statistical tools. This makes the results easy to understand, objective and actionable. This makes it a better alternative to observation based studies that are subjective and difficult to make inferences from.

Easy to understand and replicate:

Since the research provides hard figures and precise representation of the entire process, the results presented become easily comprehensible for any stakeholder. Further, it becomes easier for future researchers conducting studies around the same subject to get a grasp of prior takes on the same and replicate its results to supplement their own research.

Establishes comparison:

The presence of a control group in true experimental research allows researchers to compare and contrast. The degree to which a methodology is applied to a group can be studied with respect to the end result as a frame of reference.

Conclusive:

True experiment research combines observational and statistical analysis to generate informed conclusions. This directs the flow of follow up actions in a definite direction, thus, making the research process fruitful.

Disadvantages of Experimental Design

Expensive:

True experimental designs are costly. It takes a lot of resource investment in recruiting and managing large number of participation which is necessary for the sample to be representative. The high resource investment makes it highly important for the researcher to plan each and aspect of the process to it's minute details.

Too idealistic:

True experimental research takes place in a completely controlled environment. Such a scenario is not representative of real world situations and so the results may not be authentic. This is one of the main reasons why open field research is preferred over a lab research wherein the study can be influenced by the researcher.

Time consuming:

The process of setting up and conducting a true experiment is highly time consuming. This is because the processes like recruiting a large enough sample, gathering respondent data, random distribution into groups, monitoring the process over a span of time, tracking changes and making adjustments. The amount of processes although essential to the entire model, is not a feasible option to go for when the results are required in the near future.

The three types of true experimental designs

1) Post-test only control group design:

In this type of true experimental research, the control as well as the experimental group that has been formed using random allocation, are not tested before applying the experimental methodology. This is so as to avoid affecting the quality of the study.

The participants are always on the lookout to identify the purpose and criteria for assessment. Pre-test conveys them the basis on which they are being judged which can allow them to modify their end responses, compromising the quality of the entire research process. However, this can hinder the ability of the researcher to establish comparison between the pre-experiment and post-experiment conditions which is what actually weighs in on the changes that have taken place over the course of the research.

2) Pre-test post-test control group design:

It is a modification of the post-test control group design with an additional test carried out before the implementation of the experimental methodology. This two way testing method can help in noticing significant changes brought in the research groups as a result of the experimental intervention. There is no guarantee that the results present the true picture as post testing can be affected due to exposure of the respondents to the pre-test.

3) Solomon four group control design:

As the name suggests , this type of true experimental design involves the random distribution of sample members into 4 groups. These groups consist of 2 control groups that are not subjected to the experiments and changes, and 2 experimental groups that the experimental methodology applies to.

Out of these 4 groups, one control and one experimental group is used for pre-testing while all the four groups are subjected to post-tests.

This way researcher gets to establish pre-test post-test contrast while there remains another set of respondents that have not been exposed to pre-tests and so, provide genuine post-test responses , thus, accounting for testing effects.

Customer experience

Pre-experimental vs True experimental research design

Observatory vs Statistical:

Pre-experimental research is an observation based model i.e. it is highly subjective and qualitative in nature as against true experimental design that offers an accurate analysis of the data collected using statistical data analysis tools.

Absence vs Presence of control groups:

Pre-experimental research designs do no usually employ a control group which makes it difficult to establish contrast while all three types of true experimental designs employ control groups.

Non-randomization vs Randomization:

Pre-experimental research doesn't use randomization in certain cases whereas True experimental research always adheres to a randomization approach to group distribution.

Feasibility test vs Conclusive test:

Pre-tests are used as a feasibility mechanism to see if the methodology being applied is actually suitable for the research purpose and whether it will have an impact or not.

Steps to conduct a true experimental research

1) Identify the research objective:

Identify the variables which you need to analyse for a cause and effect relationship. Deliberate which particular relationship study will help you make effective decisions and frame this research objective one of the following manners:

Determination of the impact of X on Y
Studying how the usage/application of X causes Y

2) Identify independent and dependent variables:

Establish clarity as to what would be your controlling/independent variable and what variable would change and would be observed by the researcher. In the above samples of research purposes X is the independent variable and Y is the dependent variable.

3) Define and group population:

Define the targeted audience for the study. It is out of this target audience that a sample needs to be selected for accurate research to be carried out. It is imperative that the target population gets defined in as such detail as possible.

To narrow the field of view, a random selection of individuals from the population is carried out. These are the selected respondents that help the researcher in answering their research questions. Post their selection, this sample of individuals gets randomly subdivided into control and experimental groups.

4) Conduct Pre-tests:

Before commencing with the actual study, pre-tests are to be carried out wherever necessary. These pre-tests take an assessment of the condition of the respondent so that an effective comparison between the pre and post-tests reveal the change brought about by the research.

5) Conduct the experimental research:

Implement your experimental procedure with the experimental group created in the previous step. Provide the necessary instructions and solve any doubts or queries that the participants might have. Monitor their practices and track their progress. Ensure that the intervention is being properly complied with, otherwise the results can be tainted.

6) Conduct post-tests:

Gauge the impact that the intervention has had on the experimental group and compare it with the pre-tests. This is particularly important since the pre-test serves as a starting point from where all the changes that have been measured in the post-test, are the effect of the experimental intervention. So for example: If the pre-test in the above example shows that a particular customer service employee was able to solve 10 customer problems in two hours and the post-test conducted after a month of 2 hour workouts every day shows a boost of 5 additional customer problems being solved within those 2 hours, the additional 5 customer service calls that the employee makes is the result of the additional productivity gained by the employee as a result of putting in the requisite time

7) Analyse the collected data:

Use appropriate statistical tools to derive inferences from the data observed and collected. Correlational data analysis tools and tests of significance are highly effective relationship based studies and so are highly applicable for true experimental research.

This step also includes differentiating between the pre and the post tests for scoping in on the impact that the independent variable has had on the dependent variable. A contrast between the control group and the experimental groups sheds light on the change brought about within the span of the experiment and how such change is brought intentionally and is not caused by chance.

Explore all the survey question types possible on Voxco

FAQs

An experimental research design that uses manipulation of independent variable and random distribution of participants into groups in a controlled environment to derive cause and effect relationships through a statistical analysis.

True experimental designs are used in correlational studies where there is a need to establish cause and effect relationships. It is also used to test and compare the impact of an intervention as against a group where such intervention is absent

The three types of true experiment designs are based on the number of tests conducted: post-test only control group , pre-test post-test control group and Solomon four group control design.

There are 3 min characteristics of a true experimental design:

The presence of a control group
An independent variable to be manipulated by the researcher
Random selection and distribution of participants

A true experimental design is designed with an end objective of defining causative relationships which are often expressed as: Understanding changes caused by X on Y. These cause and effect relationship are meant to fuel effective decision making by the end user by noting if X causes Y, the degree of effect and if there are any other extraneous factors contributing to the changes in Y.

Explore Voxco Survey Software

Online page new product image3 02.png 1

What Three Factors Make a Design a True Experiment

Source: https://www.voxco.com/blog/true-experimental-design/