Sr. Data Engineer
Ventois is always looking for talented people to become its team members. We realize that productive people are always a good addition to our organization. So if you have the financial acumen and the passion to work with some of the biggest companies in the world, join in!
- Architect & build highly scalable and extensible data platforms, large scale data structures and workflows including but not limited to RDBMS / Big Data / Cloud – (AWS, GCP) which enables industrializing collection, storage, modeling, and analysis of massive data sets from heterogeneous channels that helps generate insights and addresses reporting needs.
- Develop real time and batch reliable, scalable, and highly available data pipelines from different sources and entities like third party RESTful API & databases utilizing programming languages like Python, Scala, Java, R and other technologies like Apache Spark, Hadoop, Cloud based streaming, data warehousing (Snowflake or Azure Data Warehouse) and NoSQL databases.
- Execute the development, maintenance & enhancements of data ingestion solutions of varying complexity levels and build, test and enhance data curation pipelines integration data from wide variety of data sources like DBMS, File systems (structured and unstructured), APIs and Streaming on on-premises and cloud infrastructure.
- Collaborate with Data Science teams to understand mathematical models, transform data and integrate algorithms & models into automated processes and recommend improvements to analytic methods & techniques.
- Support the development of features/attributes/inputs for the data models in an Agile manner. Host Models Via Rest APIs. Ensure non-functional requirements such as logging, authentication, error capturing and concurrency management are accounted for, when hosting models.
- Ensure the manipulation and administration of data & systems are secure and in accordance with enterprise governance by staying compliant with industry best practices, enterprise standards, corporate policy and department procedures.
- Maintain the health and monitoring of assigned data engineering capabilities that span analytic functions by triaging maintenance issues; ensure high availability of the platform; monitor workload demands; work with Infrastructure Engineering teams to maintain the data platform.
- Serve as SME for various cloud applications, understand pipelines & business metrics and develop expertise on cloud-based data stacks & pipeline development; handle the manipulation (extract, load, transform), data visualization, and administration of data and systems securely.
- Collaborate & pair with other product team members (UX, engineering & product management) to create secure, reliable, scalable software solutions; provide production application support and monitor production Service Level Objectives & Agreements (SLAs) for products.
- Perform code reviews & develop frameworks for unit, integration & end to end testing. Follow coding standards, optimize application performance & follow the latest security guidelines.
- Work with data management technologies for data integration, data quality and governance, master data management, data lineage, data catalog, metadata management and data security.
- Participate in continuous improvement activities including training opportunities, learning activities around modern software design & development core practices (communities of practice); Participate in data engineering related training, forums, summits, conferences, and certifications; Mentor junior data engineers and application developers.
Great interpersonal communication skills;
A keen eye for spotting data trends;
Great analytical skills;
A keen grasp of information technology;
Personal accountability and strong work ethic;
Professional, able to interact with vendors/clients;
Positive, “can-do” attitude.