Introducing 

Prezi AI.

Your new presentation assistant.

Refine, enhance, and tailor your content, source relevant images, and edit visuals quicker than ever before.

Loading…
Transcript

Analysis of Measurement Tool

Assessing kindergarteners’ mathematics problem solving: The development of a cognitive diagnostic test

Cognitive Diagnostic Assessment

Math problem solving (MPS) become important:

1. Student: provide oportunity for student to use content knowledge.

2. Teacher: understand the students ability of math

Azizah

  • Measures specific knowledge structures and processing skills.
  • Provides insights into cognitive strengths and weaknesses.

Previous assessment: observations, informal conversations and interviews.

Future

Instrument: Clasical Test Theory (CTT) / The Item Response Theory (IRT):

  • Total test score used to represent children's ability or achievement.
  • Ranking or set standards.
  • Differentiating passing vs. non-passing children.
  • creating and define cognitive model related to MPS
  • Develop problem solving: realife assessment

Content:

1. Reason develope of CDA

2. Previous research

3. Purpose

4. Structure and component.

5. Constructure process

6. Quality of measurement tools

7. Strengths and weakness

Assessment Tool Analysis

The purpose, structure, and construction process of the measurement tool are analyzed for assessing mathematics problem-solving ability in kindergarteners.

Quality Assessment of Measurement Tool

Strengths

Weaknesses

Item analysis is a critical component of quality assessment in measurement tools, as it helps ensure the validity, reliability, and effectiveness of the items in measuring the intended construct.

  • Mixed CDM model: selecting the most appropriate cognitive.
  • CDM: optimally use information during diagnosis, and they can obtain the interactions be- tween attributes,
  • The quality of the Q-matrix was analysed by using HCI and the psychometric characteristics to create CDT-MPS test.
  • Grouping boys and girl, rural and urban.

  • Detailed Breakdown: Breaks down math problem-solving skills into clear parts, helping teachers and parents understand children's abilities better.

  • Clear Measurement: Shows exactly which skills each test question measures, helping pinpoint why a child succeeds or struggles.

  • Hands-on Experience: Provides practical guidance for creating similar tests, offering insights into how to make them effective.
  • Starting Challenges: Early attempts faced challenges in fitting the test questions to specific models, showing the importance of getting the model right from the start.

  • Method Challenges: While the test showed good results, more research is needed to find better ways to measure skill differences when using multiple models.

  • Test Design Issues: Making real-life problems for big tests is tough, so there's still work to do in making these tests reflect everyday situations better.

Previous CDA

The Previous Precedure of CDA

1. Construct a cognitive model

Attribute structure: Children knowledge and skills to solve problem.

2. Develop cognitive diagnostic test:

  • Reveals test scores, attribute mastery patterns, and probabilities.
  • Mastery patterns indicate mastered attributes.
  • Mastery probabilities show degree of mastery.

3. Literature review:

  • Define cognitife attribute
  • Structure of cognitive attribute

Cognitive attribute and structure

4. Test Quality of Cognitive model

  • Developed an initial cognitive diagnostic test.
  • Recruited 627 children from Shanghai.
  • Examined the quality of the cognitive model.

Results:

  • Average Hierarchy Consistency Index (HCI) of 11 cognitive attributes was .749.
  • Indicates good fitting between the cognitive model and children's response data.
  • Suggests the cognitive model and Q-matrix of the test were effective.

Purpose

Intended Purpose of the Tool (CDA)

1. Assessing Kindergarteners' Mathematics Problem Solving (MPS):

  • Evaluate specific cognitive attributes and mastery patterns related to problem-solving abilities in numbers and operations.
  • Use a 0-1 scoring system to show mastery of cognitive attributes, giving a detailed assessment of children's problem-solving skills.

0: Correct response

1: Incorrect response

2. Contribution to Understanding How Kids Learn Math:

  • Offer a thorough evaluation of cognitive strengths and weaknesses in math problem-solving.
  • Analyze mastery patterns and probabilities of cognitive attributes to understand individual children's mastery levels and areas for improvement.

3. Helping Teachers Teach Better:

  • Provide teachers with insights into how kids think about math to enhance teaching methods.
  • Simplify the process for teachers to support children who need extra help in math.

Structure

The structure and componen of the measurement tool

  • Cognitive Model: The theoretical framework that outlines the knowledge and skills needed for solving math problems.

  • Cognitive Attributes: Specific aspects of mathematical thinking the test aims to assess (e.g., counting, addition).

  • Q-Matrix: A blueprint that links test items to the targeted cognitive attributes.

  • Test Development: The process of creating test items based on the Q-matrix and cognitive model.

  • Assessment Components: The different outputs of the test, including scores, mastery patterns, and probabilities, which provide detailed insights into a child's mathematical problem-solving abilities.

The cognitive attributes of Kindergartners's MPS

Construction process

The process of developing a cognitive diagnostic assessment (CDA)

1. Q-MATRIX CONSTRUCTION

Steps in constructing a Q-Matrix for assessing cognitive attributes in kindergarteners.

REACHABILITY MATRIX

Represents direct and indirect relationships among attributes.

LATENT KNOWLEDGE STATES

Reduced to eliminate irrational patterns based on attribute hierarchy.

FINAL MEASURE PATTERNS SELECTION

Removed infeasible or redundant measure patterns, ensured feasibility and understandability for kindergarteners, considered test time constraint of 30 minutes.

Measure pattern

2. the development of cognitive diagnostic test

Construction of Test Items:

a). Measure patterns from the refined Q-matrix were established first.

b). Applied problems incorporated semantic construction, mathematical relations, numerical skills, and problem-solving strategies.

Defining Cognitive Attributes

a). The research team defined the specific cognitive attributes necessary.

b). The items were adjusted to align with the defined cognitive attributes and the measure patterns in the Q-matrix.

c). This process was repeated until a consensus was reached.

Consider the difference in life experiences between urban and rural children.

Scoring criteria: these were aligned with the cognitive attributes measured by each item

3. validation procedures

EXPANSION OF Q-MATRIX

The initial Q-matrix consisted of 25 measure patterns representing different cognitive attributes, then 13 items were added based on investigation.

SCORING SYSTEM

HIERARCHY CONSISTENCY INDEX

The CDT-MPS test used a score system of 0-1, where 1 indicates attribute mastery and 0 indicates lack of mastery.

The obtained HCI value of 0.73 signifies good consistency between observed and expected response patterns from attribute hierarchy.

Topic

Learn more about creating dynamic, engaging presentations with Prezi