We are thrilled to share the results of an independent review of the SkyHive technology, affirming its robustness and fairness in skill extraction from resumes. This analysis was conducted using a test data sample that included profiles of individuals in multiple global markets including the United States, Canada, India, and the United Kingdom.
SkyHive’s Skills Models are now Armilla Verified, with the assessment showing that:
The study involved an examination where resumes were processed through the SkyHive Skill Model Inference in two distinct approaches: one anonymized, stripping all demographic details such as race, gender, age, and years of work experience, and another with these details included. The findings were consistent, showing a 97.5% overlap in skills extracted with and without demographic data, demonstrating minimal influence of such information on the results.
Furthermore, the resilience of the model was tested against irrelevant textual content embedded in resumes, such as 'lorem ipsum' text and excerpts from public domain novels. We found that the technology maintained a 95% accuracy rate in skill identification, effectively disregarding unrelated text.
The review also highlighted an area for model enhancement regarding the use of synonyms in skill descriptions. The introduction of synonyms led to a 19% reduction in overlap of identified skills. This insight will be invaluable to Skyhive as they continue to refine their patented ontology system to enhance the precision with which skills are identified and classified.
Conducted as part of SkyHive’s self-proclaimed commitment to uphold the highest standards of ethical AI use, the audit also included a bias assessment in line with New York City Local Law 144. This segment of the study focused on the 30 most-frequently extracted skills and the selection rates for candidates. The audit concluded with an "impact ratio" well within the acceptable range, providing no evidence of bias in our model. Intersectional groups were not included in the study due to insufficient data.
We congratulate the Skyhive team in voluntarily submitting their models for testing - a show of faith and confidence in their performance. This third party verification will establish credibility with their current and future clients, proving the efficacy of their models and product. Armilla AI is thrilled to be a part of their continued success.
At Armilla AI, we are dedicated to continually improving our technology to serve our users better and uphold our commitment to fairness and accuracy in talent management. This successful audit not only reaffirms the reliability of SkyHive technology but also underscores our ongoing dedication to excellence and ethical responsibility in AI development.