What on earth is SME-QA and why should you care about it?
A stunning announcement about how federal government hires recently passed largely unnoticed. Only in the age of vaccine rollout drama, impeachment, and Gamestop would this not qualify as headline news. Two weeks ago, the General Services Administration released data showing that 90% of competitive, open-to-the-public job announcements* across the federal government rely solely “on an applicant’s answers to a self-assessment questionnaire” and “an HR resume review to determine whether their experience made them eligible for the position.”
In other words: at a time when government desperately needs scientists, technologists, digital experts, and other highly specialized and skilled professionals, we have essentially one way to determine if candidates are qualified — we ask them.
Today, even someone with little or no skills could qualify for many technical jobs in government. If you know how federal hiring works, you will know how to fill out the self-assessment questionnaire you receive after you apply: just give yourself the top rating in all of the categories. No one will assess your actual skills, much less call out your deficiencies, so there’s no downside. If you know how federal hiring works, you should also know to take the keywords from the job posting and self-assessment questionnaire, and cut and paste them into your resume along with examples of what you have done in these areas. Because that is HR’s check on your self-assessment: if the HR team, who hire for all sorts of positions and may not be familiar with technical domains, see the keywords in your resume and believe there is evidence of relevant experience or education, you will be considered minimally qualified. And much of what HR looks for are the minimum qualifications from the Office of Personnel Management’s qualification standards, which for technical jobs, are qualities like attention to detail, customer service, oral communications, and problem solving. These are good qualities to have, but not what a qualified data scientist is going to put front and center on their resume.
But many truly qualified candidates use terms the HR team doesn’t know, talk about their experience and skills in ways that don’t match the job posting exactly, or submit a shorter resume, unaware that in government resumes are supposed to be many pages long. In one case, a nationally-recognized programmer applied to work for the Department of Defense. As a report from the Defense Innovation Board explains:
The classification specialist may not have known how this particular candidate’s listed experience developing “mobile applications in IonicJS, mobile applications using Angular, and APIs using Node.js, MongoDB, npm, Express gulp, and Babel,” met or did not meet the classification requirements of “experience that demonstrated accomplishment of computer-project assignments that required a wide range of knowledge of computer requirements and techniques pertinent to the position to be filled.”
This candidate, who had won a Pentagon-sponsored contest, beating out 600 other invited programmers at exactly the task the Department needed done, was initially rejected. Sadly, under the current process, the most important knowledge is how the hiring process works, not the knowledge that is needed to do the job.
If you’re saying, Wait, this can’t be how new people get hired in federal government, you’d be half right. According to GSA’s data, about half the time, no one at all is hired as the result of a competitive, open-to-the-public job announcement. That’s because while HR professionals are in charge of the qualifications process I described above, if a hiring manager believes there are no candidates in the slate HR provides who can do the job, they can reject the slate. And they do. Which means that, at a time when more and more qualified people are needed to help tackle a litany of challenges — challenges that include existential threats to our democracy and even the future of the human race — we are spending enormous amounts of time and energy not hiring anyone.
We are spending enormous amounts of time and energy not hiring anyone.
All this may be about to change, but to explain why, it helps to understand why hiring works this way now. Because the merit system is meant to be fair and neutral, HR teams in government must consider every applicant who comes through the door for every position. By contrast, in a company, if you find ten great candidates out of a large pool, you will likely consider only those ten, and not scrutinize every other resume. In government, however, you must treat everyone exactly the same — though, by law, certain categories of veterans get preference above others.
People who know the system well resent the frequent implication that it is veterans preference that makes the system ineffective. A flawed interpretation of federal hiring rules that prevents accurate assessment of candidates is what makes the system ineffective, and does a great disservice to our nation’s veterans.
But things may be changing, in part because OPM partnered with a team at the United States Digital Service (USDS) to take a closer look at the exact law that justified the incumbent hiring process. That law is Title 5 of the US Code, Section 3309, which says in part, “a preference eligible who receives a passing grade in an examination for entrance into the competitive service is entitled to additional points above his earned rating.” “Preference eligible” is used here as a noun, and it’s defined earlier in the code with a lot of legalese, but it essentially means a veteran. This section of the US code states pretty clearly that veterans preference should be applied after candidates are qualified through an examination.
So this USDS team, in conjunction with the Office of Personnel Management, has developed and productized a competitive hiring strategy that uses subject matter experts, in partnership with HR, to assess candidates. They call it SME-QA, for Subject Matter Expert Qualification Assessments. SME-QA does away with the self-assessment questionnaire. Instead, subject matter experts (SMEs) partner with HR to review resumes, using their understanding of the actual job to determine who is qualified and eligible. In other words, for a programming job, programmers are the SMEs, for a design job, its designers. To reduce bias, two SMEs independently review every resume with a third breaking any ties. When appropriate, applicants are asked to submit work samples; portfolios for designers, code samples for programmers, etc. HR trains the SMEs and checks their work to make sure all merit principles are upheld. Those who are deemed to meet the minimum requirements are then subject to up to two rounds of assessment, again, by other programmers; these can include structured interviews or written exams or demonstrations such as coding exercises. Veterans preference still applies, but only to the final list of applicants who passed the passing grade assessment hurdle.
The biggest shift here is from self-assessment qualifications (are you an expert in data science? Yes, I am!) to subject-matter qualifications (data scientists have judged you to be competent in data science) to qualify applicants as eligible for ranking and preference. This is important for all the mission-related reasons described above, and it’s important for equity. Today, unless someone tells you how the system works (usually someone already in government) your application may stand little chance of making the cut. This means that a system supposedly designed to be fair and neutral in reality advantages those with access and insider knowledge. It also advantages current feds, since veterans preference does not apply to them. These dynamics reinforce existing biases and replicate current demographics.
Perhaps more importantly, research shows that marginalized groups are less likely to overstate their qualifications, putting them at a disadvantage in a system that almost insists you do so. In order to truly pursue equity, it is crucial that the panels of experts who will assess candidates be diverse themselves. USDS achieved this in their first few pilots, but leaders will need to ensure that this practice and others continue for the full equity benefits to accrue.
One carrot for agencies, ironically, is reduced workload. It sounds more burdensome to involve SMEs in the process, but it’s not, for three important reasons. First, postings stand a much better chance of succeeding in making a hire if the candidates put forth are qualified. SME-QA adoption should track directly with a reduction in failed postings (which again, is currently half.) Second, the SME-QA process is designed to work especially well for specialized positions with multiple vacancies for the same role. In other words, if there is a high need for many data scientists in one agency, or across agencies, a single hiring process can attract hundreds of candidates, and result in multiple hires through the same process. This exact scenario has already played out to huge success in testing the SME-QA process government-wide. In another pilot with the Department of Interior, SME-QA cut the days to make a candidate selection from 45 to 16. This is one of those cases where better is also faster.
The SME-QA process is better for veterans, too. By applying veterans preference after the assessment, hiring managers will trust that all veterans on the slates HR provides are fully qualified for the job. Instead of throwing out the entire slate, as they so often do, they’ll hire the veterans first, and because of multi-agency hiring, then they’ll hire the non-veterans as well. It’s a win-win for both candidates and government.
Sadly, better and faster does not mean that SME-QA is destined to become the new normal for federal hiring. Quite the opposite. Despite the severity of this problem and the compelling evidence of SME-QA’s success in fixing it, changing the hiring practices of tens of thousands of HR professionals across government is no easy task. OPM must be crystal clear and boldly assertive in insisting, not just suggesting, that agencies empower hiring managers and SMEs to work with HR to develop assessments for critical specialized jobs.
We should never have gotten to the point where half of all hiring for competitive positions fail. Thousands of qualified professionals apply to the federal government every week, people whose skills our nation desperately needs, and a new administration is making government service even more exciting to many. We can’t afford to keep turning them away because of misguided notions about hiring rules.
*Open to the public, competitive job announcement excludes hiring through special authorities. Many technical people today are hired through special hiring authorities.
