How can threats to internal and external validity be addressed

Answer:

Internal validity refers specifically to whether an experimental treatment/condition makes a difference or not, and whether there is sufficient evidence to support the claim.

External validity refers to the generalisation of treatment/condition outcomes.

Explanation:Greatest threat to Internal Validity :

History--the specific events which occur between the first and second measurement.

Maturation--the processes within subjects which act as a function of the passage of time. i.e. if the project lasts a few years, most participants may improve their performance regardless of treatment.

Testing--the effects of taking a test on the outcomes of taking a second test.

Instrumentation--the changes in the instrument, observers, or scorers which may produce changes in outcomes.

Statistical regression--It is also known as regression to the mean. This threat is caused by the selection of subjects on the basis of extreme scores or characteristics. Give me forty worst students and I guarantee that they will show immediate improvement right after my treatment.

Selection of subjects--the biases which may result in selection of comparison groups. Randomization (Random assignment) of group membership is a counter-attack against this threat. However, when the sample size is small, randomization may lead to Simpson Paradox, which has been discussed in an earlier lesson.

Experimental mortality--the loss of subjects. For example, in a Web-based instruction project entitled Eruditio, it started with 161 subjects and only 95 of them completed the entire module. Those who stayed in the project all the way to end may be more motivated to learn and thus achieved higher performance.

Selection-maturation interaction--the selection of comparison groups and maturation interacting which may lead to confounding outcomes, and erroneous interpretation that the treatment caused the effect.

John Henry effect--John Henry was a worker who outperformed a machine under an experimental setting because he was aware that his performance was compared with that of a machine.

Greatest threat to External Validity:

Interaction of testing and X--because the interaction between taking a pretest and the treatment itself may effect the results of the experimental group, it is desirable to use a design which does not use a pretest.

Interaction of selection and X--although selection is controlled for by randomly assigning subjects into experimental and control groups, there remains a possibility that the effects demonstrated hold true only for that population from which the experimental and control groups were selected. An example is a researcher trying to select schools to observe, however has been turned down by 9, and accepted by the 10th. The characteristics of the 10thschool may be vastly different than the other 9, and therefore not representative of an average school. Therefore in any report, the researcher should describe the population studied as well as any populations which rejected the invitation.

How to address internal threat:

History--this is controlled in that the general history events which may have contributed to the O1 and O2 effects would also produce the O3 and O4 effects. This is true only if the experiment is run in a specific manner--meaning that you may not test the treatment and control groups at different times and in vastly different settings as these differences may effect the results. Rather, you must test simultaneously the control and experimental groups. Intrasession history must also be taken into consideration. For example if the groups truly are run simultaneously, then there must be different experimenters involved, and the differences between the experimenters may contribute to effects.

A solution to history in this case is the randomization of experimental occasions--balanced in terms of experimenter, time of day, week and etc.

Maturation and testing--these are controlled in that they are manifested equally in both treatment and control groups.

Instrumentation--this is controlled where conditions control for intrasession history, especially where fixed tests are used. However when observers or interviewers are being used, there exists a potential for problems. If there are insufficient observers to be randomly assigned to experimental conditions, the care must be taken to keep the observers ignorant of the purpose of the experiment.

Regression--this is controlled by the mean differences regardless of the extremety of scores or characteristics, if the treatment and control groups are randomly assigned from the same extreme pool. If this occurs, both groups will regress similarly, regardless of treatment.

Selection--this is controlled by randomization.