Looking for a skilled PySpark Developer? Persystent Consultancy is hiring. Join our team and work on exciting big data projects. Apply now!
If interested, Click here to share your Resume or mail your CV at [email protected]
Job details
Salary
10 to 16 LPA
Experience
3 to 5 Years
Qualifictions
Bachelor's (Preferred)
Skill
PySpark, Big Data Processing, Data Manipulation, Spark Ecosystem, Python, SQL, Performance Optimization
Location
Noida
Joining Time
Immediate to 15 Days
Job Description
Roles And Responsiblties :-
- 5+ years of experience in programming with python. Strong proficiency in python.
- Familiarity with functional programming concepts.
- 3+ years of Hands on experience in developing ETL data pipelines using pyspark on AWS EMR.
- Hands on experience of JSON processing using python.
- Good understanding of Spark’s RDD API.
- Good understanding of Spark’s Dataframe and API.
- Experience in configuring EMR clusters on AWS.
- Experience and good understanding of Apache Spark Data sources API.
- Experience of dealing with AWS S3 object storage from Spark.
- Experience in trouble shooting spark jobs. Knowledge of monitoring spark jobs using Spark UI.
- Performance tuning of Spark jobs.
- Understanding fundamental design principles behind a business processes.
- Design, development and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR.
- Writing reusable, testable, and efficient code.
- Integration of data storage solutions in spark – especially with AWS S3 object storage.
- Performance tuning of pySpark scripts.
- Need to ensure overall build delivery quality is good and on time delivery is done at all times.
- Should be able to handle meetings with customers with ease.
- Need to have excellent communication skills to interact with customer.
Please Note
Knowledge of AWS SDK CLI