Skip to content
SquareHRWired SquareHRWired SquareHRWired
SquareHRWired SquareHRWired SquareHRWired
  • Recruiting
    • Talent Acquisition
    • Applicant Tracking and Recruiting Software
    • Video Interviewing Software
    • Recruitment Marketing
  • Engagement
    • Corporate Portals
    • Surveys
    • Employee Recognition
    • Internal Communications (IC) Platforms
  • Learning
    • Learning Management Systems
    • eLearning
    • Digital Adoption Platforms
    • Skills Development Platforms
  • Pay & Benefits
    • Employee Time Tracking Software
    • Payroll Software
    • Earned Wage Access (EWA) Platforms
    • Employee Financial Wellness Tools
  • HR Tech
    • HR Analytics Software
    • Cloud HCM
    • Workforce Planning
    • HR Automation
  • HR Compliance
    • FLSA & Wage Compliance
    • Data Privacy Management
    • OSHA & Workplace Safety
    • Labor Law
  • Future of Work
    • Remote & Hybrid Work
    • Workplace Culture
    • Diversity, Equity & Inclusion (DEI)
    • AI in HR
  • Recruiting
    • Talent Acquisition
    • Applicant Tracking and Recruiting Software
    • Video Interviewing Software
    • Recruitment Marketing
  • Engagement
    • Corporate Portals
    • Surveys
    • Employee Recognition
    • Internal Communications (IC) Platforms
  • Learning
    • Learning Management Systems
    • eLearning
    • Digital Adoption Platforms
    • Skills Development Platforms
  • Pay & Benefits
    • Employee Time Tracking Software
    • Payroll Software
    • Earned Wage Access (EWA) Platforms
    • Employee Financial Wellness Tools
  • HR Tech
    • HR Analytics Software
    • Cloud HCM
    • Workforce Planning
    • HR Automation
  • HR Compliance
    • FLSA & Wage Compliance
    • Data Privacy Management
    • OSHA & Workplace Safety
    • Labor Law
  • Future of Work
    • Remote & Hybrid Work
    • Workplace Culture
    • Diversity, Equity & Inclusion (DEI)
    • AI in HR
Close

Search

  • https://www.facebook.com/
  • https://twitter.com/
  • https://t.me/
  • https://www.instagram.com/
  • https://youtube.com/
SquareHRWired SquareHRWired SquareHRWired
SquareHRWired SquareHRWired SquareHRWired
  • Recruiting
    • Talent Acquisition
    • Applicant Tracking and Recruiting Software
    • Video Interviewing Software
    • Recruitment Marketing
  • Engagement
    • Corporate Portals
    • Surveys
    • Employee Recognition
    • Internal Communications (IC) Platforms
  • Learning
    • Learning Management Systems
    • eLearning
    • Digital Adoption Platforms
    • Skills Development Platforms
  • Pay & Benefits
    • Employee Time Tracking Software
    • Payroll Software
    • Earned Wage Access (EWA) Platforms
    • Employee Financial Wellness Tools
  • HR Tech
    • HR Analytics Software
    • Cloud HCM
    • Workforce Planning
    • HR Automation
  • HR Compliance
    • FLSA & Wage Compliance
    • Data Privacy Management
    • OSHA & Workplace Safety
    • Labor Law
  • Future of Work
    • Remote & Hybrid Work
    • Workplace Culture
    • Diversity, Equity & Inclusion (DEI)
    • AI in HR
  • Recruiting
    • Talent Acquisition
    • Applicant Tracking and Recruiting Software
    • Video Interviewing Software
    • Recruitment Marketing
  • Engagement
    • Corporate Portals
    • Surveys
    • Employee Recognition
    • Internal Communications (IC) Platforms
  • Learning
    • Learning Management Systems
    • eLearning
    • Digital Adoption Platforms
    • Skills Development Platforms
  • Pay & Benefits
    • Employee Time Tracking Software
    • Payroll Software
    • Earned Wage Access (EWA) Platforms
    • Employee Financial Wellness Tools
  • HR Tech
    • HR Analytics Software
    • Cloud HCM
    • Workforce Planning
    • HR Automation
  • HR Compliance
    • FLSA & Wage Compliance
    • Data Privacy Management
    • OSHA & Workplace Safety
    • Labor Law
  • Future of Work
    • Remote & Hybrid Work
    • Workplace Culture
    • Diversity, Equity & Inclusion (DEI)
    • AI in HR
Close

Search

  • https://www.facebook.com/
  • https://twitter.com/
  • https://t.me/
  • https://www.instagram.com/
  • https://youtube.com/
Home/Learning/How L&D Teams Can Test Course-Evaluation Analytics With Synthetic Data

How L&D Teams Can Test Course-Evaluation Analytics With Synthetic Data

By Sophie Williams
November 21, 2026

Learning and development teams can test whether analytics tools uncover meaningful patterns in course feedback by working with a simulated dataset whose underlying issues are known in advance. In an article published November 19, Zsolt Olah, a senior HR data measurement analyst at Intel, describes how he built such a dataset to compare conventional text analytics, generative AI and a hybrid approach. His central recommendation is to use code for dependable calculations and data handling, while applying AI and human review to interpret open-ended comments.

Build a dataset with known patterns

The simulated company in Olah’s experiment, Helix Microsystems, had 1,200 learners and 10 courses delivered in self-paced, classroom and virtual formats over six months. The files represented courses, sessions, learner profiles, enrollments, completions, assessment results and survey responses. The dataset contained 1,386 responses, reduced to 1,380 after duplicates were removed. Because the data was fictional, the experiment could include a separate answer key describing the patterns that each analytical method was expected to find.

Olah recommends beginning with a clear account of the organization’s actual evaluation process: the survey questions, what each question is intended to measure and the challenges that arise when interpreting responses. Distinctive short IDs for questions can make instructions easier to apply. The simulated data should reflect realistic learner groups and course-specific problems, rather than just produce a large volume of generic comments. Olah’s examples included night-shift technicians unable to attend live sessions, new managers who liked a course but were not ready to apply its content, and experienced employees assigned to beginner-level training.

The design also included deliberately difficult data. Examples included sarcasm, comments that mixed praise with criticism, non-English responses, duplicate submissions, repeated rating patterns and personal contact details. Some groups experiencing the greatest problems were made less likely to respond, reflecting a potential weakness in feedback data: survey results may not represent employees who face the largest barriers. Olah said the first version of the comments was too repetitive, causing conventional text analysis to group responses by repeated wording instead of meaning. Rewriting the comments to vary their language made the test more realistic.

Compare methods against a scoring rubric

The experiment evaluated three approaches: AI working directly from the data, conventional statistical and text analysis run in code, and a hybrid process combining code with AI-assisted comment analysis. To make the comparison more consistent, Olah used a 40-point rubric. It assigned 22 points to identifying 11 planted patterns with the correct learner group and cause, 12 points to handling data traps, and six points to reliability, including repeatability and accurate counts and quotations. Unsupported claims lost points. The answer key was kept from the methods until their findings were ready to be assessed.

Conventional analysis scored 32 out of 40. It identified the planted patterns largely by comparing results across groups such as shift, business unit, facilitator and region, and its counts were exact. It also detected duplicates, identical-answer response patterns and contact details. But its interpretation of comments was less reliable: word groupings matched the intended themes only loosely, sarcasm was treated as positive, and it often failed to preserve both sides of comments that contained praise and criticism.

Generative AI performed better at reading tone and identifying themes, but its counts and group attribution were less dependable. Across six runs, Claude Opus 5.5 scored between 38 and 39, while Gemini 4 Aragon scored between 33 and 37. In 10 of 12 runs, the systems noticed that some learners in course C07 rated their own ability highly, but often connected the pattern to the wrong group or did not identify the group. Olah also reported that Gemini calculated course scores using between 29% and 54% of responses without disclosing that coverage, producing differences of as much as 15 percentage points. One run that had Gemini generate its own code used about two million tokens and consumed a $20 credit.

Implications for HR and learning teams

Olah’s preferred hybrid workflow assigns numerical work to Python: counting, averaging, sorting, deduplication and personal-information handling. AI is then used to help interpret comments, with a human approving an initial set of themes before the system labels responses in batches. In his experiment, the AI first reviewed a sample of 400 comments to propose themes; after human review, it applied an approved pool of 20 to 30 themes to the full set. The findings were then tied to numbers calculated in code, rather than relying on the AI to generate its own figures.

For HR and L&D teams, the experiment is a reminder that learner satisfaction alone does not establish whether training is useful. Olah’s prompts prioritize evidence connected to employees’ ability to apply learning, and call for themes to be weighed across the number of learners and courses reporting them rather than by the intensity of an individual comment. The exercise is a controlled test, not proof that any one method will perform identically on an employer’s live data. Teams would need to adapt the survey, learner groups and known issues to their own context, and use tools approved for work data. Olah also cautions that AI systems may still expose names or other identifying details despite instructions to remove them. His results point to a measured approach: test methods on data with known answers, verify the calculations, protect sensitive comments and retain human oversight of interpretation.

Tags:

generative-ailearning-analyticssynthetic-survey-data
Author

Sophie Williams

Follow Me
Other Articles
Previous

Career Development Emerges as the Primary Driver for Employee Retention and Recruitment

Next

Knowledge Sharing Can Turn Employee Expertise Into Organizational Capability

No Comment! Be the first one.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Popular 🔥

  • The Evolution of Talent Acquisition Careers: A 2026 Perspective
  • Understanding the Distinct Roles of Change Management and Employee Onboarding
  • Examining the Impact of Artificial Intelligence on Candidate Experience
  • EEOC Secures $500K Settlement Over Male-Only Job Openings
  • United Cinemas Adopts Workforce.com to Streamline Time Tracking and Operations

Archives

  • November 2026
  • October 2026

Categories

  • Engagement (44)
  • Future of Work (42)
  • HR Compliance (42)
  • HR Tech (42)
  • Learning (42)
  • Pay & Benefits (42)
  • Recruiting (44)
SquareHRWired

SquareHRWired covers HR, workforce management, workplace technology, talent, employee experience, and the future of work.

  • Facebook
  • X
  • LinkedIn

Company

About Us
Contact

Categories

Recruiting
Engagement
Learning
Pay & Benefits
HR Tech
HR Compliance
Future of Work

Copyright © 2026 SquareHRWired.All rights reserved.
Privacy Policy | Terms | Cookie Policy | Do Not Sell My Personal Info

SquareHRWired is a part of SalesboxAI.