Research Competencies Statement

The following serves as an introduction to the research portion of my portfolio that can be found at the bottom of this page.  The purpose of this introduction is to provide a link between the research goals described in my candidate’s statement, the competencies I need to pursue these goals, evidence of my progress towards achieving said competencies, as well as to outline specific short term goals that will enable me to continue to build my research competencies.

My Scholarly Efforts & My Research Agenda

In my candidate’s statement I described my research agenda as being focused on making contributions to the field of Instructional Systems Technology (IST) through scholarly work on technology enhanced formative and summative assessment systems for use in the teaching profession.  At this point in my scholarly career I have transitioned from a class focus to a research focus.  I have completed my course work and conducted several independent research projects on the use of educational technology in the classroom.

My Philosophy of Research

My philosophy of research and my present perspective on how research should be conducted has been heavily influenced by Dr. Frick through his EDUC R 690.  His article entitled “Types of Knowledge of Education Created through Disciplined Inquiry” was a central text that informed EDUC R 690 and helped me form my philosophy of research.  I agree with his assertion that there is no single correct approach or method that should be used for research.  Therefore, I am inclined not to align myself only with qualitative or quantitative research methodologies, but instead consider mixed method approaches as they are likely to lead to more thorough answers to research questions.

I am most comfortable with quantitative methods and frequently try to develop computational ways to analyze a given situation and identify patterns.  My quantitative leanings originate from my computer science background and employment with Sapient where numbers, logic, and the bottom line ruled.  My academic experiences, especially my qualitative methods class, helped to temper my quantitative leaning with exposure to methods that created valuable knowledge that could not be revealed using quantitative methods.

Ideally a researcher should have the ability to evaluate the quality of research conducted using any established research method.  Being able to evaluate the quality of a broad range of research requires that a researcher understand the strengths and limitations of a large variety of research methods in order to be able to judge if the claims being made in a particular research study can be supported with the evidence gathered via the methods used.  The methods used in a research study also need to be able to be evaluated against established standards and conventions.  A broad understanding of research methods also helps the researcher pick appropriate research tools when designing approaches to answer their own research questions.

An extensive understanding of research methods needs to be complemented by an exhaustive understanding of a handful of specific research methods.  A balance between breadth and depth is required so that the researcher can evaluate a broad range of research but still expertly design and implement research studies.  As it is impossible to be an expert on every research method, a researcher needs to identify a smaller subset of methods most suitable for their research agenda.  Specialization allows for more skillful and efficient application of these research methods to answer specific research questions.

Identifying worthwhile research questions requires more than a strong background in research methodology: a researcher also needs to understand the current state of knowledge in their chosen research area.  To understand the current state of knowledge in a given area, a balance between general and specialized knowledge is necessary.  A broad perspective is needed to keep on top of developments related to a general research area.  A deep understanding of a specialized research area is needed to ensure that the research questions have not already been answered and are based on the knowledge generated by other research studies.

Research Competencies I Will Need

Given my research agenda and career goals I will need to deepen my research competencies in several specific areas to complement a broader set of research skills.  I need to achieve a more in-depth familiarity with the state of knowledge in the general field of Instructional Systems Technology and specifically in my two areas of focus: analysis and evaluation; design and development.  Within these broad areas I need to have a specific understanding of the literature related to variable length computerized classification testing, computer adaptive testing, integration of technology into classrooms, methods of assessment, academic integrity (specifically plagiarism) and teaching technology.

Another set of research competencies I need to work on involve the communication of research results and interaction with the larger research community through publishing in established peer reviewed journals and presenting at professionally recognized national conferences.

Research Competencies I Have

The following will describe the set of research skills for which I have achieved the target level of competency.  Below I will outline my role in the production of research related artifacts and how the experiences have helped me develop specific research competencies.

The two independent research projects that I conducted in the spring of 2008 (see “NETS-T Multiple-Choice Measure of Educational Technology Ability” and “Formative Research on Implementing Peer Formative Assessment in a Preservice Teacher Technology Integration Learning Classroom Community”) facilitated the improvement of my research competencies.  Both independent research projects focus on assessment in educational technology, but while one is based on formative assessment and peer feedback, the other concentrates on summative survey based measures.

The formative assessment project was a collaboration between myself and Craig Howard, a fellow EDUC W201 assistant instructor.  Craig took the lead on obtaining IRB approval and I addressed the problem of how to allow students to provide anonymous formative feedback to each other in the Oncourse course management system.  We both worked on the research design and exploration of the relevant literature.  Several sections of EDUC W201 represented our population and in the end we had nearly eighty students participate in the research.  The primary purpose of the research was to investigate how students communicate peer feedback to each other in an online environment under two different conditions; one where student provide feedback anonymously and the other where student identities were known by all.  Using the results from this research project, Craig and I have been able to present multiple times at national conferences and author a manuscript that has been accepted by the Journal of Educational Computing Research.

My second independent research project conducted in the spring of 2008 has resulted in an article that has been accepted for publication with revisions by TechTrends.  This research was conducted by myself and involved the creation of a multiple-choice survey measure of educational technology ability that align with the new National Educational Technology Standards for Teachers (NETS-T).  I created and deployed an online survey to undergraduate and graduate students in the School of Education to gather demographic data, information about respondents’ educational technology school experiences, self-reported measures of educational ability, and responses to the multiple-choice NETS-T survey measure.  Over two hundred students responded to the survey and I was able to find significant positive correlations between the NETS-T survey measure and a variety of other indicators of educational technology ability.

My competency in reviewing and evaluating research is demonstrated through two literature reviews.  The most recent was produced independently in the spring of 2008 as part of EDUC R650 (see “Balancing Varied Assessment Functions To Promote Exemplary Educational Technology Integration: A Review of Literature”).   The second was produced independently in the spring of 2006 as part of R711 (see “Improving Evaluation Through the Broader Application of Criterion Reference Testing”). Both literature reviews reflect my continuing interest in assessment and evaluation in a variety of learning contexts.

Another competency that I have developed is the production of article and book critiques.  The first, “Rather Long Notes on the ‘Notes on the Synthesis of Form by Christopher Alexander’” was created in the fall of 2005 for R690.  The second, “Management Critique: ‘The Fifth Discipline’” was written in the spring of 2007 for SPEA V602.  Both critiques focus on higher-level theories of the design of systems and reflect my interest in systems thinking and it’s application to educational contexts.

Producing research proposals is a proficiency I have made progress on throughout my doctoral studies.  The research proposal titled “A Qualitative Inquiry into How Key Stakeholders Define Group Project Success in Professional Oriented Masters Degree Programs” that I presented in the spring of 2007 for R695 was written independently and also used to meet a course requirement for R711.  This proposal reflects my interest in service learning and the use of project management techniques in the context of group projects.

Proof of my progress in developing my ability to create presentations for national conferences is provided by the acceptance of all four of my submitted proposals to AECT in 2009 and both of my proposals to AERA in 2010.  I was lead presenter on two of the AECT presentations and second and third on the remaining two.  While in 2009 I was second presenter on a single presentation at AERA, in 2010 I will be lead presenter on two presentations.  Each of these presentations stem from my two independent research project that I will discuss in greater detail below.  My previous presentations at the IST conferences since 2006 and the feedback I received prepared me to write better conference proposals as well as to deliver higher quality presentations.

My other conference presentations include the 2006, 2007, 2008, and 2009 IST Conferences.  My first presentation, “Paper Prototyping a Dynamic Software Interface: A Case Study Using APT” was presented with Jamison Judd and it was based on the work we did as part of R690 and the associated research report.  The second presentation, “Learning and Teaching Technology Integration” was presented with other members of Dr. Brush’s research group at the Spring 2007 IST Conference and it primarily concentrated on the work of Jung Won Hur who had written an AECT presentation proposal on the topic.  I coordinated the efforts to translate this work into a presentation for the IST conference.  Jung Won Hur, Theano Yerasimou, Ying Wang, Claudius Rodgers, and I participated in the final presentation.  My 2008 and 2009 IST Conference presentations resulted from work with Craig Howard on the independent research projects previously discussed.  Both Craig and I shared the responsibility for the creation and presentation of the research at the IST Conference.

In addition, I independently presented at the 2008 North American Association of Environmental Education (see “Leveraging ICT to Promote Multidisciplinary Environmentally Focused Authentic Educational Projects”; “Sustainability Now Versus Later: Adult vs. Child Environmental Education” and “The Problems (And Opportunities) Related to Assessing Environmental Literacy”).

Producing independent research project reports is another area I have developed a strong competency in.  My first report titled “Paper Prototyping a Dynamic Software Interface: a Case Study Using APT” was created in the fall of 2005 for R690 in partnership with Jamison Judd and reported on the development of a prototype software interface for Dr. Frick’s analysis tool called Analysis of Patterns in Time.  During the usability testing of this prototype qualitative and quantitative data was collected on the user’s ability to successfully complete a series of tasks.  Participants were then interviewed to gather additional information about their experience with the prototype.  The report presents the data gathered and suggests possible interface changes that could be designed into a future version of the interface.  This research report demonstrates competencies related conducting observations and interviews and analyzing both qualitative and quantitative data gathered.  Jamison Judd and I worked closely together on all aspects of this project.

The second project report is called “Collaborative Study: Students’ Perceived Costs and Benefits of Pursuing the Doctoral Degree at Indiana University” and was created in the spring of 2005 for R611 in a group project. I produced “Individual Findings and Implications” as part of the same research.  In this project we collectively came up with our research questions, conducted a  review of related literature, and designed a structured interview protocol.  Each member of the group then independently used the interview protocol to conduct and transcribe an interview.  Members of the group also conducted observations in a variety of contexts that would also inform the research questions.  The observation I conducted was during a meeting between several IST students and a member of ETS.  The interview transcripts and observation notes were shared with other group members and each member independently produced a separate findings and implications document.  I gained experience planning, conducting, and analyzing interviews and observations through my involvement in this research.

Research Competencies I Still Need

I need to continue building on the research skills and experience that I have developed through my classwork and independent research.  I see my dissertation contributing heavily to my progress on both my research agenda and my goals related to my scholarship.

Supporting Evidence

Publications

Presentations

  • Barrett, A. (2010). Measuring TPK component of TPACK: An Alternative to Self-Assessment. To Be Presented at the American Educational Research Association Conference, Denver, Colorado.
  • Barrett, A., & Howard, C. (2010). Validity of Computer Mediated Formative Peer Assessment: Pre-service Teacher’s Comments in Asynchronous CMC. To Be Presented at the American Educational Research Association Conference, Denver, Colorado.
  • Barrett, A. (2009). Development of an Assessment to Measure Teachers Educational Technology Knowledge. Presented at the Association for Educational Communications and Technology Conference, Louisville, Kentucky.
  • Barrett, A., & Howard, C. (2009). Product Quality in Online Peer Assessment: Pre-service Teacher’s Comments in Asynchronous CMC. Presented at the Association for Educational Communications and Technology Conference, Louisville, Kentucky.
  • Frick, T., Howard, C., Barrett, A., Enfield, J., & Myers, R. (2009). Alternative Research Methods:MAPSAT Your Data to Prevent Aggregation Aggravation. Presented at the Association for Educational Communications and Technology Conference, Louisville, Kentucky.
  • Howard, C., & Barrett, A. (2009a). Anonymous Discussions: How learners commented on each others’ work in a protected space. Presented at the Association for Educational Communications and Technology Conference, Louisville, Kentucky.
  • Howard, C., & Barrett, A. (2009b). Anonymity in Online Peer Reviews: Preservice Teachers’ Online Comments in Two Critique Designs. Presented at the American Educational Research Association Annual Meeting Conference, San Diego, CA.
  • Leveraging ICT to Promote Multidisciplinary Environmentally Focused Authentic Educational Projects presented at the NAAEE (North American Association for Environmental Education) 2008 conference
  • Sustainability Now Versus Later: Adult vs. Child Environmental Education presented at the NAAEE 2008 conference
  • The Problems (And Opportunities) Related to Assessing Environmental Literacy presented at the NAAEE 2008 conference
  • Anonymity and Wikis: Learners’ formative feedback in a pre-service teacher technology course presented with Craig Howard at the Spring 2008 IST Conference
  • Learning and Teaching Technology Integration presented with other members of Dr. Brush’s research group at the Spring 2007 IST Conference
  • Paper Prototyping a Dynamic Software Interface: A Case Study Using APT presented with Jamison Judd at the Spring 2006 IST Conference

Independent Research

Independent Research Project Reports

Articles Submitted to Peer Reviewed Journals

Literature Reviews

Critiques

Research Proposals

  • Print
  • del.icio.us
  • Digg
  • Facebook
  • Google Bookmarks
  • Twitter