연구방법론에 대해서 아래와 같이 어느 한 reviewer로부터 엄청나게 몰매를 맞은 적이 있습니다:
——
Experimental Design & Method-
There is no way to know how students used the materials. Did they really read the text AND look at the diagrams? How do we know that they didn’t just read the text or didn’t just look at the diagrams? Without some kind of measurement of what they did (e.g., eye tracking, text-only or diagrams-only control conditions), we have no idea about the process, or how spatial ability might play a role.
Study time is not a very interesting dependent variable if we don’t know what the students were doing during that time. It certainly cannot be used to measure cognitive load without having some other type of data to confirm that students took longer because their load was higher.
Satisfaction data doesn’t really tell us anything about cognition.
The use of multiple choice questions makes it hard to assess the representations that students built about the system. It’s hard to infer much of anything about students’ mental representations from a multiple choice test. Was the test validated? If someone who had not seen the materials took the test, how many correct answers would they get?
There were no examples of the materials in the manuscript. The reader has no idea what the diagrams looked like, or where the text was placed in relation to the diagrams. Were the materials piloted, or what did the authors do to ensure that they followed good instructional design?
——
Measurement(측정방법)과 measure(측정값)에 대해서 지적하고 있습니다.
우리가 체중계를 사용해서 몸무게를 잰다고 할 때, 체중계는 measurement이고 체중의 값은 measure입니다.
위의 reviewer는 제가 어떠한 변수들의 값을 측정하기 위해서 사용한 측정방법이 옳지 않다는 지적을 하고 있습니다.
또한 변수들의 값을 측정하였다고 하더라도, 그 값의 결과를 가지고 제 연구가설이 옳다고 해석할 수 있는 근거가 없다고 말합니다.
마지막으로, 제가 사용한 measurement 방법의 예시를 투고한 논문에서는 전혀 보여주지 않았다고 언급하고 있습니다.
