Vanderbilt University and the Vanderbilt Libraries are in an exciting time of transformation. With new leadership at the Chancellor, Provost and University Librarian levels, the libraries are re-envisioning the future of library services at Vanderbilt. Through radical collaboration we seek to deepen student and faculty engagement with the libraries, equip students for life-long information proficiency, increase faculty research impact, create more opportunities for interdisciplinary experimentation, and optimize access to collections and resources, including marginalized and hidden histories and cultures.

We are searching for a dynamic individual to join our team, someone who is interested in unique datasets and driven to support cutting-edge academic research. We are seeking a forward-thinking and solution-oriented professional eager to collaborate and innovate.

The Data Engineer is an integral part of the Vanderbilt Libraries’ Television News Archive and digital lab and plays a pivotal role in enhancing research capabilities. This individual will be instrumental in crafting and overseeing the processes that organize and make accessible large datasets, particularly from the Television News Archive, thereby empowering scholarship, and discovery at the university.

Duties and Responsibilities

• Create streaming pipelines to ingest data daily from the TV News Archive.

• Integrate machine learning, language models, and other natural language processing tools into the metadata creation process for the TVNA.

• Lead the ELT process for the library’s data lake for existing datasets.

• Move datasets from external sources to AWS S3 in preparation for loading them into the data lake.

• Develop, document, and regularly update data ingestion pipelines in Databricks and out to TV News Archive access portals.

• Develop strategies for enhancing data from the TV News Archive, and other special collections.

• Coordinate with internal teams at the Vanderbilt Libraries to develop dashboards and other data quality tools.

• Keep track of developments in Databricks and recommend new approaches to data management and quality assurance.

Qualifications

• A Bachelor’s Degree in computer science, data science, applied mathematics, or similar or equivalent work experience is required.

• A minimum of 2 years’ working with ELT/ETL pipelines is required.

• Ability to manage multiple priorities in a changing environment.

• Experience with large data cleanup and/or migration projects is preferred.

• Experience using Apache Spark with Python or Scala is preferred.

• Experience using Amazon Web Services, especially Comprehend, Transcribe, Lambda, or Glue is preferred.

Commitment to Equity, Diversity, and Inclusion

At Vanderbilt University, we are intentional about and assume accountability for fostering advancement and respect for equity, diversity, and inclusion for all students, faculty, and staff. Our commitment to diversity makes us who we are. We have created a community that celebrates differences and lets individuality thrive. As part of this commitment, we actively value diversity in our workplace and learning environments as we seek to take advantage of the rich backgrounds and abilities of everyone. The diverse voices of Vanderbilt represent an invaluable resource for the University in its efforts to fulfill its mission and strive to be an example of excellence in higher education.

Vanderbilt University is an equal-opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran, or any other characteristic protected by law.

Please note, all candidates selected for an offer of employment are subject to pre-employment background checks, which may include but are not limited to, based on the role for which they have been selected: criminal history, education verification, social media review, motor vehicle records, credit history, and professional license verification.

#LI-ONSITE

Commitment to Equity, Diversity, and Inclusion At Vanderbilt University, we are intentional about and assume accountability for fostering advancement and respect for equity, diversity, and inclusion for all students, faculty, and staff. Our commitment to diversity makes us who we are. We have created a community that celebrates differences and lets individuality thrive. As part of this commitment, we actively value diversity in our workplace and learning environments as we seek to take advantage of the rich backgrounds and abilities of everyone. The diverse voices of Vanderbilt represent an invaluable resource for the University in its efforts to fulfill its mission and strive to be an example of excellence in higher education. Vanderbilt University is an equal opportunity, affirmative action employer. Women, minorities, people with disabilities, and protected veterans are encouraged to apply.