SPARK Developer
Ventois is always looking for talented people to become its team members. We realize that productive people are always a good addition to our organization. So if you have the financial acumen and the passion to work with some of the biggest companies in the world, join in!
Project Location(s): Work location is Shrewsbury, MA with required travel to client locations throughout the USA.
Education: Bachelor’s Degree in Computer Science, Computer Information Systems, Information technology, or a combination of education and experience equating to the U.S. equivalent of a Bachelor’s degree in one of the aforementioned subjects.
Responsibilities
- Perform development for a Hadoop based architecture using Map Reduce, YARN, HDFS, HBase, Hive, Pig, Oozie, Apache Spark for ingestion.
- Write processes in Spark to Ingest data from various sources including sftp, mainframe, RDBMS, Kafka etc.
- Participate in data migration from relational databases to Hadoop HDFS.
- Write Spark code to transform data using business rules provided and load Hive and RDBMS tables.
- Translate, load and present disparate data sets in multiple formats/sources including JSON.
- Develop policies and procedures related to development and support of new and ongoing systems to ensure the integrity of data.
- Participate in analysis of data stores and help with data analytics.
- Collaborate with product development teams and senior designers to develop architectural requirements to ensure client satisfaction with the product.
- Participate in Application performance tuning and troubleshooting.
- Ability to work with a variety of teams and clients, mentoring junior resources, and effectively handling multiple tasks/deadlines with success.
- Excellent communication and documentation skills, especially when communicating with business users.
- Leadership skills, motivation to keep abreast of emerging technologies and pick up new skill sets quickly.
Skills/Experience
Great interpersonal communication skills;
A keen eye for spotting data trends;
Great analytical skills;
A keen grasp of information technology;
Professional demeanor;
Personal accountability and strong work ethic;
Professional, able to interact with vendors/clients;
Positive, “can-do” attitude.