Technical Manual · v1 (2026)

Assessment Methodology

This page documents how DataU Upskillers assessments are designed, scored, and interpreted. It exists to enable independent verification by employers, universities, government bodies, and international organisations.

Validity Statement

DataU Upskillers assessments are formative, framework-aligned indicators for career development planning. They have not yet undergone criterion-related validity studies (predicting job performance), Differential Item Functioning (DIF) analysis, or ISO 17024 personnel certification audit. Scores should be used as one signal among many in hiring or admissions decisions — not as standalone criteria.

View the public State of Skills report →

International Framework Alignment

Every question in our assessments is aligned to specific competency codes in published international standards. This provides content validity evidence — the questions cover the domain they claim to measure.

AI Literacy Assessment

FrameworkReferenceRole in our assessment
DigComp 2.2 (EU, 2022)doi.org/10.2760/115376Primary content validity reference. Questions aligned to specific area codes (1.1–5.4) and proficiency levels (1–8).
OECD AI Competency Framework (2025)ailiteracyframework.orgBasis for PISA 2029 AI literacy module. Used for domain categorisation (Engage/Create/Manage/Design).
UNESCO AI Competencies for Students (2024)UNESCO.org12 student competencies covering ethical AI, safety, and human-centred skills. Referenced for ethics and risk questions.
Meta AI Literacy Scale (MAILS)Carolus et al., 2023Psychometric inspiration for item design. Not a licensed implementation.

Power Skills (Soft Skills) Assessment

FrameworkReferenceRole in our assessment
NACE Career Readiness Competencies (2021)naceweb.orgPrimary employer-defined competency taxonomy. All 8 NACE competency areas (CA_1–CA_8) covered across the full assessment.
O*NET Content Model v29.0 (2025)onetcenter.orgOccupational competency element IDs (e.g., 4.A.4.b.2) cited per question. Provides traceability to DOL employment data.
SHRM Behavioral Competency Framework (2022)shrm.org9 HR-professional competencies aligned to our soft skill categories.
PMI Power Skills (2023)pmi.orgProject management profession alignment for leadership and problem-solving questions.
O*NETThis product uses O*NET Content Model data (v29.0) and the O*NET Occupational Database, developed by the U.S. Department of Labor, Employment and Training Administration (USDOL/ETA). O*NET is a trademark of USDOL/ETA.

Assessment Design

AI Literacy (v3, 2026)

  • · 5-item quick screen / 21-item full instrument
  • · 5 competency categories: Generative AI, Risks & Ethics, Tools & Productivity, Data Privacy, Future of Work
  • · 3 difficulty levels (1=Foundational, 2=Applied, 3=Expert)
  • · Polytomous scoring: 0 (wrong) / 5 (plausible) / 10 (correct)
  • · Correct-answer distribution: a×6, b×5, c×6, d×4 — balanced to prevent response-set bias
  • · Format: 4-option MCQ with one clearly best answer per OECD item design guidelines

Power Skills (v3, 2026)

  • · 5-item quick screen / 26-item full instrument
  • · 5 categories: Collaboration, Agility, Leadership, Problem Solving, EQ
  • · Format: Situational Judgement Test (SJT) — more valid than trait self-report (McDaniel et al. 2001 meta-analysis)
  • · Polytomous scoring: 0 / 5 / 10 — partial credit for situationally reasonable responses
  • · Correct-answer distribution: a×7, b×6, c×6, d×6 — balanced
  • · All scenarios grounded in NACE/O*NET/SHRM competency definitions

Bias controls: Distractors are written to be plausible (not obviously absurd) to prevent test-wiseness effects. Scenarios avoid role, gender, or cultural stereotypes. Cultural relevance for SEA contexts (hybrid teams, AI adoption, cross-cultural work) is embedded in scenario framing.

Talent Index Formula

The Talent Index is a composite index (0–100) combining four empirically-grounded sub-factors. It is not a validated psychometric instrument — it is a transparent, auditable career readiness indicator.

Formula:

Core = (Si + Cx + Eq) / 3

TalentIndex = Core × Dampener(Av)

Dampener(Av) = 1 / (1 + (Av/75)^1.8), floor 0.38

Si — Social Intelligence: log-scale proxy from CV soft-skill count; replaced by O*NET avgTaskSocial when Atlas is available. Source: O*NET Research Center; Thorndike (1920); Cantor & Kihlstrom (1985).

Cx — Cognitive Complexity: log-scale proxy from skill breadth; replaced by O*NET avgTaskCognitive. Source: Savickas & Porfeli (2012) Career Adapt-Abilities Scale.

Eq — Emotional Quotient: formal Power Skills assessment score (primary); log-scale CV proxy if unavailable. Source: Bar-On EQ-i model (2006); Goleman (1998).

Av — Automation Velocity: O*NET/Atlas task automation percentage (primary); LLM risk proxy (secondary). Source: Brynjolfsson & Mitchell (2017); Frey & Osborne (2017).

Equal weighting of Si, Cx, Eq: justified by Dawes (1979) no-differential-weights principle — appropriate in absence of calibration data.

Score Interpretation & AQRF Mapping

Scores map to the ASEAN Qualifications Reference Framework (AQRF) — the 8-level regional standard used by all 10 ASEAN member states for qualification equivalence. This mapping makes scores meaningful to SEA employers, governments, and universities.

AI Literacy Score → AQRF Level

Score

0–24

AQRF 1

Foundational — 1–2 (Foundation)

Requires substantial AI onboarding; suitable for roles with minimal AI exposure

OECD level: Engage: Recognise

Score

25–44

AQRF 2

Developing — 2–3 (Foundation→Intermediate)

Can use mainstream AI tools (ChatGPT, Copilot) for routine tasks with guidance

OECD level: Engage: Understand

Score

45–59

AQRF 3

Competent — 3–4 (Intermediate)

Independently uses AI tools; can automate routine workflows; needs oversight for high-stakes outputs

OECD level: Engage: Apply + Create: Assist

Score

60–74

AQRF 4

Proficient — 4–5 (Intermediate→Advanced)

Strong AI collaborator; can lead team AI adoption; ready for AI-augmented professional roles

OECD level: Create: Produce + Manage: Govern

Score

75–87

AQRF 5

Advanced — 5–6 (Advanced)

AI champion / project lead; can design and implement AI solutions for business problems

OECD level: Create: Produce + Design: Architect

Score

88–100

AQRF 6

Expert — 7–8 (Highly Specialised)

AI strategist / technical lead; defines organisational AI roadmap; CTO/AI Director adjacent

OECD level: Design: Innovate + Create: Lead

AQRF Level Descriptors (Source: ASEAN 2nd Ed., 2021)

Level 1: Entry
Knowledge: General basic knowledge · Autonomy: Under direct supervision in a structured context
Level 2: Elementary
Knowledge: Basic factual knowledge of a field · Autonomy: Under supervision with some autonomy
Level 3: Pre-Intermediate
Knowledge: Knowledge of facts, principles, processes and general concepts · Autonomy: Take responsibility; adapt own behaviour to circumstances
Level 4: Intermediate
Knowledge: Factual and theoretical knowledge in broad contexts · Autonomy: Self-management within guidelines; manage others in simple contexts
Level 5: Upper-Intermediate
Knowledge: Comprehensive specialised factual and theoretical knowledge · Autonomy: Management and supervision in unpredictable contexts
Level 6: Advanced/Degree
Knowledge: Advanced knowledge with critical understanding of theories and principles · Autonomy: Manage complex professional activities with full autonomy
Level 7: Expert/Postgraduate
Knowledge: Highly specialised knowledge at the forefront of a field · Autonomy: Manage complex and unpredictable work contexts
Level 8: Master/Doctoral
Knowledge: Knowledge at the most advanced frontier of a field · Autonomy: Substantial authority, innovation, autonomy, scholarly integrity

Statistical Properties & Reliability

Reliability Standards Referenced

  • · α ≥ 0.90 — Excellent; suitable for high-stakes individual decisions
  • · α ≥ 0.80 — Good; professional development decisions
  • · α ≥ 0.70 — Acceptable; formative/screening use
  • · α < 0.70 — Marginal; treat with caution
  • · Source: Nunnally (1978); AERA/APA/NCME Standards (2014)

Standard Error of Measurement

SEM = SD × √(1 − α) (Allen & Yen, 1979). A score of 72 with SEM=6 means the user's true score falls in 72 ± 6 with 68% probability, or 72 ± 12 with 95% probability. Displayed on all score pages.

Current reliability data

Cronbach's alpha is computed from live response data as the platform grows. Minimum N=20 complete response matrices required for meaningful α. Current status and live reliability statistics are available at /api/assessments/reliability.

Target: α ≥ 0.75 for AI Literacy (formative use); α ≥ 0.75 for Power Skills (formative use). Future goal: α ≥ 0.80 for professional development, α ≥ 0.90 for high-stakes screening.

Adoption by Employers, Universities & Governments

Employers

  • Use as a structured screening signal alongside CV and interview
  • Score-to-AQRF mapping gives consistent interpretation across candidates
  • Framework alignment (NACE/O*NET) lets you verify question content independently
  • Never use as the sole hiring criterion (validity studies in progress)
  • Request /api/assessments/reliability for current statistical properties

Universities

  • Assessments are aligned to DigComp 2.2 and OECD AI framework — internationally recognised
  • Suitable for student development planning and digital readiness benchmarking
  • Not a substitute for academic assessment or institutional accreditation
  • Partner with us for concurrent validity study (link our scores to your GPA/graduate outcomes)
  • AQRF mapping allows cross-ASEAN qualification comparison

Governments & Ministries

  • Score bands map to AQRF levels 1–6 (aligned to all ASEAN member states)
  • Cambodia NQF, Indonesia KKNI, Thailand TQF alignment via AQRF cross-reference
  • Suitable for workforce development programme benchmarking
  • Not a replacement for national qualification certification
  • Contact us for MOU on workforce intelligence data sharing

Institutional Enquiries

For validation partnerships, concurrent validity studies, government MOU discussions, or employer integration queries, contact our research team:

research@wedatau.org

DataU Upskillers Assessment Methodology · v3 · Published 2026-01-01 · Phnom Penh, Cambodia