Considerations when designing studies

Ken Kelley and Scott E. Maxwell state:

“At a minimum, the following points must be considered when designing studies in the behavioral, educational, and social sciences:

  1. the question(s) of interest must be determined;
  2. the population of interest must be identified;
  3. a sampling scheme must be devised;
  4. selection of independent and dependent measures must occur;
  5. a decision regarding experimentation versus observation must be made;
  6. statistical methods must be chosen so that the question(s) of interest can be answered in an appropriate and optimal way;
  7. sample size planning must occur so that an appropriate sample size given the particular scenario, as defined by points a through f, can be used;
  8. the duration of the study and number of measurement occasions need to be considered;
  9. the financial cost (and feasibility) of the proposed study calculated.”

Ken Kelley and Scott E. Maxwell, Sample Size Planning with Applications to Multiple Regression: Power and Accuracy for Omnibus and Targeted Effects, [Kelley2008] http://nd.edu/~kkelley/publications/chapters/Kelley_Maxwell_Chapter_SSMR_2008.pdf Links to an external site.

[Kelley2008] Ken Kelley and Scott E. Maxwell, ‘Sample Size Planning with Applications to Multiple Regression: Power and Accuracy for Omnibus and Targeted Effects’, in The SAGE handbook of social research methods, Pertti Alasuutari, Ed. Newbury Park, CA, USA: SAGE Publications Ltd, 2008, pp. 166–192 [Online]. Available: http://nd.edu/~kkelley/publications/chapters/Kelley_Maxwell_Chapter_SSMR_2008.pdf Links to an external site.

Transcript

So, Kelley and Maxwell said, when you're designing a study at the minimum, you have to consider the following points when you're doing behavioral, educational, and social sciences study: What is the question of interest?  What is the population of interest?  What is the sampling scheme that we're going to apply?  What are going to be our independent and dependent measures?  Should we do experiments or simply observations?  What statistical methods are we going to choose so that we can actually answer the question in an appropriate and optimal way?  We don't bother more people than we really need to - to be able to get the data that we're interested in.  And of course, we need to make sure that we actually have a sufficient sampling plan, so were going to get the data that we need. We need to think about how long the study can take place. So, for example, in this course, a very important thing is you need to be able to get it done in such a timely fashion that you're able to actually go and analyze the data and produce the final report within the time period for this course.  And of course, in most practical cases, you are going to have to think about the financial cost and the feasibility of the study that you're going to propose before you start to do it.  Because otherwise, you're not going to be able to complete it.