Workhuman’s Sruthi Nair discusses her role as a quality assurance engineer and what it’s like to work in a male-dominated field.
“As one can imagine, testing data projects can be challenging, but this is also what makes it really interesting,” says Sruthi Nair, a quality assurance (QA) engineer in Workhuman’s data team.
Nair, who supports the functional and technical work associated with several data projects, notes that her job requires her to be “meticulous”, while providing opportunities for learning and growth.
“With the changing needs of data consumers, it becomes very important to redefine test criteria based on the varying business specifications. Requirements like these allow us to be more engaged and invested in the work we do.”
Nair’s responsibilities as a QA engineer include providing timely feedback on the quality of code that is shipped on a regular basis and validating different aspects of data quality. “Essentially, it involves ensuring the overall quality of the data pipelines developed and owned by the data team.”
‘You should never limit yourself from taking risks, trying new opportunities and preparing for change’
What does a typical day look like for you?
During any typical day my responsibilities vary between collaborating with data engineers, reviewing their code and supporting their work, and reaching out to product owners and data-dependent teams to clearly understand business specifications.
An important part of my work involves preparing the test environment and setting up the required test data, strategising, defining test plans and data quality metrics for different projects and creating test suites.
Additionally, performing test execution, automation and data validations depending on the specific needs of each project and troubleshooting issues that may arise in different pipelines, are all part of the job.
What types of projects do you work on?
During my time with Workhuman, I have had a great opportunity to be a part of a multitude of projects. I’ve worked and been involved with numerous data ingestion pipelines, ETL/ELT projects including validating end-user datasets and data reporting pipelines, DBT-based data transformation projects and Workhuman’s Data API project, amongst others.
What skills do you use on a daily basis?
In addition to tools that facilitate testing that include frameworks like Pytest and Behave, libraries that are useful for data validation like Pandas and Pandera, and the use of container-based simulated test environments, my role also requires skills and tools that are extensively used in data engineering.
These include SQL, data transformation tools like DBT, data processing frameworks like Spark, orchestration tools like Airflow, AWS services – Redshift, S3, Glue, Cloudwatch, DMS etc, infrastructure as code tools. My team mainly uses the Python programming language.
However, the most useful skills for a QA to have are the ability to assess risk appropriately and have a good understanding of the needs and perspective of customers and end users. It is essential to possess good analytical and troubleshooting skills. Additionally, effective communication and the ability to clearly present quality concerns and issues are fundamental to drive the team towards greater success.
What is the hardest part of your working day?
As a data QA engineer, it is difficult but also highly crucial to set the right quality standards and benchmarks against which the quality of the data and the effectiveness and reliability of the pipelines that deliver the required data can be measured.
As multiple teams are dependent on the work delivered by the data engineering team, it is necessary to address issues, respond effectively to changes across different data sources, and be conscious of changing business specifications of the data needs of end users.
This adds up to the multitude of projects that the data team is required to be involved with and more often than not, the QA needs to be actively engaged in many of these areas. The involvement of QA in many such cases becomes crucial to help avoid the future occurrence of issues and often provides opportunities to revisit and redefine test criteria and any previously defined expectations.
Therefore, it’s absolutely important to make the right decisions while trying to effectively organise and prioritise work each day to make the most of the work hours.
When you first started this job, what were you most surprised to learn was important in the role?
Although I had a fairly good amount of experience working as a QA engineer, I’m relatively new to the experience of working as a QA in a data team. I do feel this role is often less understood in the industry. While working on data, I feel it is essential to work closely with the data engineers and be able to understand their language and familiarise yourself with the technical work they do.
Here, I find myself more involved in the development process than I have ever been before.
In my opinion, a siloed QA team could never work effectively when it comes to testing data and the pipelines that are responsible for delivering data needs. So, I find myself constantly learning new technologies and reviewing code, as these are crucial to be more effective as a QA working with data. I’m really lucky to have very helpful colleagues who are very generous with sharing knowledge and extending help to anyone who is in need.
As the data team offers a platform that serves as a single source of truth of data for the whole organisation, there are several internal teams, analysts and customers who rely on this data in various forms. Therefore, it becomes all the more important to set the right benchmarks for ensuring data quality and this demands for the involvement of multiple key members across teams to be highly invested in helping to define these data standards.
The model that is followed here is the one of shared responsibility. This also boosts strong collaboration between all individuals in our team and even with others.
How has this role changed as this sector has grown and evolved?
The role of a QA engineer is changing and I see the shift-left approach of testing getting very popular across many organisations. And I feel this is a really positive advancement in the field of quality, wherein QAs are required to be involved quite early in the software developmental life cycle and contribute to writing unit tests.
Testing early enables the team to identify issues well ahead of time and it becomes less costly to fix them. As a result, the overall quality of the product is improved. The needs for shift-left testing become more pronounced in data teams as the early involvement of QA helps to make them more aware of business specifications, customer expectations and how the technical design and implementation align with these needs. Oftentimes, I reflect on how I can impart more value to the data engineering team, and testing early facilities this, while also ensuring that the test scripts and QA metrics are well-aligned with business needs.
Moreover, data is continually getting bigger, the number of sources is increasing resulting in more variety of data. All this is happening at a very fast rate. With the adoption of cloud-based solutions to serve these needs, tools used earlier have now become obsolete. So, you have to make consistent efforts to catch up with the evolving requirements of the job.
Platform teams, in particular data, tend to be male-dominated. How has your experience been in this field?
I’m really grateful that I’m in a team with highly supportive and talented colleagues who are very appreciative of my work. I’m also fortunate that I work for a company that values and promotes gender diversity. I feel well respected and accepted. My contributions and viewpoints are highly valued. I’m considered an important member and I don’t feel judged based on my gender. But I do understand that working conditions may not be as favourable for women everywhere and in such cases, they might have to go through the struggle of asserting their needs on a regular basis.
As would be the case in any male-dominated industry, it is understandable why women would feel uncertain to make a start or transition into a career in data, particularly on the engineering side. In many situations like these, women could get too self-critical and may feel the overwhelming need to prove their self-worth.
But in my opinion, this can make us second guess ourselves to an unreasonable extent that some of us might let go of potential job opportunities due to a contorted and diminished view of our true capabilities. Some could also feel discouraged by a few failed attempts. Irrespective of gender, believing in oneself and trusting one’s abilities is the first step towards preparing for the journey ahead.
I feel what matters the most is that data teams generally attract diligent and committed individuals with an active learner’s mindset, who are passionate in embracing constant change that is characteristic of the evolving data world. You should never limit yourself from taking risks, trying new opportunities and preparing for change no matter what stage you might be at in your career.
10 things you need to know direct to your inbox every weekday. Sign up for the Daily Brief, Silicon Republic’s digest of essential sci-tech news.
