Evaluate and assess AI models in furtherance of the AI Action Plan.
Design methods for scientifically sound evaluations of AI systems.
Evaluate AI systems in application, post-deployment, and in real world settings.
Develop methods, tooling, and guidelines for rigorous evaluation of end-to-end AI systems in use (e.g., in the workplace and/or in sector-specific tasks) to identify failure modes and areas of unreliability.
Develop analyses of evaluations and AI measurement instruments themselves, for example, evaluation and improvement of red teaming, benchmarks, and automated scoring techniques.
Adapt and improve evaluation techniques, and develop new techniques where needed.
Collaborate with colleagues and across teams to develop innovative solutions to AI evaluation problems.
Communicate complex AI concepts to both technical and non-technical audiences.
Inform the development of technical guidelines and best practices for AI evaluation.
Males born after 12-31-59 must be registered for Selective Service
Suitable for Federal employment
Completion of Confidential Financial Disclosure, Form OGE-450
Designated and/or random drug testing required
Bargaining Unit Position: No
Qualifications
All applicants must submit a resume NOT TO EXCEED two pages on standard-size paper. If the resume exceeds two pages, only the first two pages will be reviewed for qualifications or assessment determinations.
Basic Requirements:
Experience must be IT related; the experience may be demonstrated by paid or unpaid experience and/or completion of specific, intensive training (for example, IT certification), as appropriate.
For all positions individuals must have IT-related experience demonstrating each of the four competencies listed below. The employing agency is responsible for identifying the specific level of proficiency required for each competency at each grade level based on the requirements of the position being filled.
Attention to Detail - Is thorough when performing work and conscientious about attending to detail.
Customer Service - Works with clients and customers (that is, any individuals who use or receive the services or products that your work unit produces, including the general public, individuals who work in the agency, other agencies, or organizations outside the Government) to assess their needs, provide information or assistance, resolve their problems, or satisfy their expectations; knows about available products and services; is committed to providing quality products and services.
Oral Communication - Expresses information (for example, ideas or facts) to individuals or groups effectively, taking into account the audience and nature of the information (for example, technical, sensitive, controversial); makes clear and convincing oral presentations; listens to others, attends to nonverbal cues, and responds appropriately.
Problem Solving - Identifies problems; determines accuracy and relevance of information; uses sound judgment to generate and evaluate alternatives, and to make recommendations.
ZP-2210-IV: In addition to the basic requirements, applicants must have one year (52 weeks) of specialized experience equivalent to at least the GS-12 level (ZP-III at NIST). Specialized experience is defined as experience demonstrating technical understanding of Artificial Intelligence (AI) generative models and Machine Learning (ML) algorithms; designing scientifically sound technical evaluation methods in real settings; carrying out rigorous technical evaluations to identify functionality and reliability issues; and collaborating and coordinating on AI-related projects.
ZP-2210-V: In addition to the above requirements, applicants must have one year (52 weeks) of specialized experience equivalent to at least the GS-14 level (ZP-IV at NIST). Specialized experience is defined as experience demonstrating advanced technical understanding of Artificial Intelligence (AI) generative models and Machine Learning (ML) algorithms; designing scientifically sound technical evaluation methods in real settings; carrying out rigorous technical evaluations to identify functionality and reliability issues; and leading cross-functional AI-related teams and projects. Establishes best practices with specific technical standards and workflows.
Experience refers to paid and unpaid experience, including volunteer work done through National Service programs (e.g., Peace Corps, AmeriCorps) and other organizations (e.g., professional; philanthropic; religious; spiritual; community, student, social). Volunteer work helps build critical competencies, knowledge, and skills and can provide valuable training and experience that translates directly to paid employment. You will receive credit for all qualifying experience, including volunteer experience.
The qualification requirements in this vacancy announcement are based on the U.S. Office of Personnel Management (OPM) Qualification Standards Handbook.
If requesting reconsideration of your qualification determination, please refer to the following site: Applicant Reconsideration
Education
Qualifications are based upon meeting specialized experience only. Education cannot be used to qualify for this position.
Use of foreign education for qualifications. An accredited organization must evaluate education completed outside of the U.S. to ensure that it is comparable to education received in accredited institutions in the U.S. Click here to view a listing of accredited organizations from the Department of Education's website. A copy of the foreign education evaluation (containing the results with course-by-course listing) is required with your application.
Additional information
This position is covered under NIST's Alternative Personnel Management System (APMS), a pay-for-performance system with excellent HR flexibilities to help NIST recruit and retain top talent Learn more about the APMS here!
We may share your application package with other selecting officials at NIST with opportunities like this one. Additional selections may be made through this vacancy.
Click all links in this vacancy announcement to view additional information or instructions.
You MUST select at least ONE ELIGIBILITY to be considered for this position.
All documents submitted for this announcement must be legible to make qualification or eligibility determinations.
This appointment may be extended up to four years without further competition.
If the selection results in a temporary promotion, it can be extended not to exceed five years or made permanent without further competition.
This position requires the passing of a drug test.
This position requires filing an OGE-450 Confidential Financial Disclosure Report within 30 days of onboarding and annually thereafter.
As a condition of employment for accepting this position, you may be required to serve a 1-year trial period during which we will evaluate your fitness and whether your continued employment advances the public interest. In determining if your employment advances the public interest, we may consider:
- your performance and conduct;
- the needs and interests of the agency;
- whether your continued employment would advance organizational goals of the agency or the Government; and
- whether your continued employment would advance the efficiency of the Federal service.
Upon completion of your trial period your employment will be terminated unless you receive certification, in writing, that your continued employment advances the public interest.
Candidates should be committed to improving the efficiency of the Federal government, passionate about the ideals of our American republic, and committed to upholding the rule of law and the United States Constitution.
A career with the U.S. government provides employees with a comprehensive benefits package. As a federal employee, you and your family will have access to a range of benefits that are designed to make your federal career very rewarding. Opens in a new windowLearn more about federal benefits.
Eligibility for benefits depends on the type of position you hold and whether your position is full-time, part-time or intermittent. Contact the hiring agency for more information on the specific benefits offered.
How you will be evaluated
You will be evaluated for this job based on how well you meet the qualifications above.
All applicants must submit a resume NOT TO EXCEED two pages on standard-size paper. If the resume exceeds two pages, only the first two pages will be reviewed for qualifications or assessment determinations.
How you will be evaluated: We will review your resume, optional cover letter and supporting documents to assess your qualifications for this position. Please be sure that your resume and supporting documents include detailed information to support your qualifications for this position. Failure to provide sufficient and/or legible evidence in your resume or supporting documents may result in a "not qualified" determination. You MUST select at least ONE eligibility (i.e., how you wish to be considered) to receive consideration for this position.
Referral: All applicants who meet the minimum qualifications will be referred to the selecting official for consideration. Under Direct Hire Authority, applicants are not rated and ranked, and veteran preference does not apply.
CTAP or ICTAP eligibility: If you are a displaced or surplus Federal employee eligible for the Career Transition Assistance Plan (CTAP) or Interagency Career Transition Assistance Plan (ICTAP), you must be determined eligible and rated as well-qualified to receive special selection priority. "Well Qualified" means that in addition to meeting the minimum qualification requirements and selective placement factor(s), if applicable, you must receive a score of 85 or higher. Please review the following site regarding required documents to support CTAP or ICTAP eligibility and additional information regarding this authority - CTAP/ICTAP Eligibility
A career with the U.S. government provides employees with a comprehensive benefits package. As a federal employee, you and your family will have access to a range of benefits that are designed to make your federal career very rewarding. Opens in a new windowLearn more about federal benefits.
Eligibility for benefits depends on the type of position you hold and whether your position is full-time, part-time or intermittent. Contact the hiring agency for more information on the specific benefits offered.
A complete application includes: 1) A resume (not to exceed two pages), 2) vacancy question responses regarding technical qualifications, and 3) submission of any required documents. Please note that if you do not provide all required information, as specified in this announcement, or if the documents submitted are illegible you may not be considered for this position (or may not receive the special consideration for which you may be eligible).
Resume: All applicants must submit a resume by creating one in USAJOBS or uploading one of their choosing. The resume MUST NOT exceed two pages on standard-size paper. If the resume exceeds two pages, only the first two pages will be reviewed for qualifications or assessment determinations. Cover letters are optional. To receive any credit for relevant experience, please list the month/year and the number of hours worked for experience listed on your resume. We suggest you review the qualification requirements for this position as you may need to customize your resume to ensure that it supports the position qualifications. Please refer to the Resume Guidance on how to build an effective resume.
If you are relying on your education to meet qualification requirements:
Failure to provide all of the required information as stated in this vacancy announcement may result in an ineligible rating or may affect the overall rating.
Please read the entire announcement and all the instructions before you begin an application. To apply and be considered for this position, you must complete all required questionnaires, assessments, and required documentation as specified in the How to Apply and Required Documents section. The complete application package must be submitted by 11:59 PM (ET) on the closing date of the announcement to receive consideration. The application process is as follows:
Click the Apply Online button.
Answer the questions presented in the application and attach all necessary supporting documentation.
Click the Submit Application button prior to 11:59 PM (ET) on the announcement closing date.
As part of the application process, you will be asked to respond to four narrative questions. While your responses to these questions is not required and will not be scored, we encourage you to thoughtfully address each question. You will be asked to certify that you are using your own words and did not use a consultant or artificial intelligence (AI) such as a large language model (LLM) like ChatGPT or Copilot.
Once your application has been submitted, you will receive a confirmation notification by email. The Human Resources Office will evaluate your application to determine your eligibility for the position. After the evaluation is complete, you will receive another notification regarding the status of your application.
To update your application, including supporting documentation: During the announcement open period, return to your USAJOBS account, find your application record, and click Edit my application. This option will no longer be available once the announcement has closed.
To view the announcement status or your application status: Click on this: https://www.usajobs.gov/Help/how-to/application/status/. Your application status page is where you can view your application status, USA Hire assessment completion status, and review your notifications sent by the hiring agency regarding your application.
Need help applying?
If you are experiencing any difficulties with the online application process (anything after you have clicked the 'Apply Online' button), please contact the Agency Contact listed on this announcement between regular business hours (8:00 a.m. - 4:00 p.m. ET). If you receive any kind of error message, please be ready to provide a screenshot or, at a minimum, the error message text and number. We will provide you with a technical support number for further assistance if necessary.
If you are having trouble with USAJOBS (searching for jobs, account password/profile issues, or Resume Builder), please visit the USAJOBS Resource Center. We regret that we are unable to support any issues with USAJOBS, as this service is not maintained by the NIST.
If you are experiencing a significant hardship that hinders your ability to apply online, please contact the Agency Contact listed on this announcement prior to the vacancy closing date and during regular business hours to discuss your situation.
NIST Office of the Director
100 Bureau Drive
Building 101/Room A1137
Gaithersburg, MD 20899
US
Next steps
To verify the status of your application, log into your USAJOBS account (https://my.usajobs.gov/Account/Login), all of your applications will appear on the Welcome screen. The Application Status will appear along with the date your application was last updated. For information on what each Application Status means, visit: https://www.usajobs.gov/Help/how-to/application/status/.
We will update applicant statuses when:
A qualification determination is made. If found not qualified/eligible, no additional updates will be provided.
Applications are referred to the selecting official.
A selection decision has been made.
Please notify us if your contact information changes after announcement's closing date. Also, note that if you provide an inaccurate email address or if your mailbox is full or blocked (e.g., spam-blocker), you may not receive important communication, which could affect your consideration for this position.
The Federal hiring process is set up to be fair and transparent. Please read the following guidance.
A complete application includes: 1) A resume (not to exceed two pages), 2) vacancy question responses regarding technical qualifications, and 3) submission of any required documents. Please note that if you do not provide all required information, as specified in this announcement, or if the documents submitted are illegible you may not be considered for this position (or may not receive the special consideration for which you may be eligible).
Resume: All applicants must submit a resume by creating one in USAJOBS or uploading one of their choosing. The resume MUST NOT exceed two pages on standard-size paper. If the resume exceeds two pages, only the first two pages will be reviewed for qualifications or assessment determinations. Cover letters are optional. To receive any credit for relevant experience, please list the month/year and the number of hours worked for experience listed on your resume. We suggest you review the qualification requirements for this position as you may need to customize your resume to ensure that it supports the position qualifications. Please refer to the Resume Guidance on how to build an effective resume.
If you are relying on your education to meet qualification requirements:
Failure to provide all of the required information as stated in this vacancy announcement may result in an ineligible rating or may affect the overall rating.
Please read the entire announcement and all the instructions before you begin an application. To apply and be considered for this position, you must complete all required questionnaires, assessments, and required documentation as specified in the How to Apply and Required Documents section. The complete application package must be submitted by 11:59 PM (ET) on the closing date of the announcement to receive consideration. The application process is as follows:
Click the Apply Online button.
Answer the questions presented in the application and attach all necessary supporting documentation.
Click the Submit Application button prior to 11:59 PM (ET) on the announcement closing date.
As part of the application process, you will be asked to respond to four narrative questions. While your responses to these questions is not required and will not be scored, we encourage you to thoughtfully address each question. You will be asked to certify that you are using your own words and did not use a consultant or artificial intelligence (AI) such as a large language model (LLM) like ChatGPT or Copilot.
Once your application has been submitted, you will receive a confirmation notification by email. The Human Resources Office will evaluate your application to determine your eligibility for the position. After the evaluation is complete, you will receive another notification regarding the status of your application.
To update your application, including supporting documentation: During the announcement open period, return to your USAJOBS account, find your application record, and click Edit my application. This option will no longer be available once the announcement has closed.
To view the announcement status or your application status: Click on this: https://www.usajobs.gov/Help/how-to/application/status/. Your application status page is where you can view your application status, USA Hire assessment completion status, and review your notifications sent by the hiring agency regarding your application.
Need help applying?
If you are experiencing any difficulties with the online application process (anything after you have clicked the 'Apply Online' button), please contact the Agency Contact listed on this announcement between regular business hours (8:00 a.m. - 4:00 p.m. ET). If you receive any kind of error message, please be ready to provide a screenshot or, at a minimum, the error message text and number. We will provide you with a technical support number for further assistance if necessary.
If you are having trouble with USAJOBS (searching for jobs, account password/profile issues, or Resume Builder), please visit the USAJOBS Resource Center. We regret that we are unable to support any issues with USAJOBS, as this service is not maintained by the NIST.
If you are experiencing a significant hardship that hinders your ability to apply online, please contact the Agency Contact listed on this announcement prior to the vacancy closing date and during regular business hours to discuss your situation.
NIST Office of the Director
100 Bureau Drive
Building 101/Room A1137
Gaithersburg, MD 20899
US
Next steps
To verify the status of your application, log into your USAJOBS account (https://my.usajobs.gov/Account/Login), all of your applications will appear on the Welcome screen. The Application Status will appear along with the date your application was last updated. For information on what each Application Status means, visit: https://www.usajobs.gov/Help/how-to/application/status/.
We will update applicant statuses when:
A qualification determination is made. If found not qualified/eligible, no additional updates will be provided.
Applications are referred to the selecting official.
A selection decision has been made.
Please notify us if your contact information changes after announcement's closing date. Also, note that if you provide an inaccurate email address or if your mailbox is full or blocked (e.g., spam-blocker), you may not receive important communication, which could affect your consideration for this position.
Fair and transparent
The Federal hiring process is set up to be fair and transparent. Please read the following guidance.
