Data Modeler

Ventois is always looking for talented people to become its team members. We realize that productive people are always a good addition to our organization. So if you have the financial acumen and the passion to work with some of the biggest companies in the world, join in!

Project Location(s): Work location is Shrewsbury, MA with required travel to client locations throughout the USA.
Education: A minimum of a Bachelor’s degree in computer science, computer information systems, information technology, or a closely related field, or a combination of education and experience equating to the U.S. equivalent of a Bachelor’s degree in one of the aforementioned subjects.

Responsibilities

  • Configure Spark streaming to get ongoing information from Kafka and store the stream information to HDFS.
  • Collect data using Spark Streaming from AWS S3 bucket in near real time and perform necessary Transformation and Aggregations to build the data model and persist the data in HDFS.
  • Involve in creating data pipelines and CI/CD deployment pipelines.
  • Define and execute appropriate steps to validate various data feeds in Hadoop.
  • Create partitions and buckets for the tables in the Hive data warehouse. Create topics for the respective tables from the source system.
  • Involve in migrating the data from on-premises to AWS cloud platform.
  • Optimize Hive QL/pig scripts by using execution engines like Tez, Spark.
  • Coordinate with the business team to get the requirements and desired business outcome.
  • Develop spark code using Scala and Spark SQL for processing data and faster testing.
  • Implement Spark using Scala and utilizing Data frames and Spark SQL API for faster processing of data.
  • Work closely with the team within Agile methodology.

Skills/Experience

  • Great interpersonal communication skills;
  • A keen eye for spotting data trends;
  • Great analytical skills;
  • A keen grasp of information technology;
  • Professional demeanor;
  • Personal accountability and strong work ethic;
  • Professional, able to interact with vendors/clients;
  • Positive, “can-do” attitude.
This website uses cookies and asks your personal data to enhance your browsing experience.