84 việc làm mới nhất toàn quốc

Key Responsibilities:

1. Crawler Development and Maintenance:

• Design, develop, and maintain reliable web crawlers for data collection from social media platforms.
• Optimize crawling algorithms to ensure accurate and efficient data retrieval while adhering to ethical standards and platform policies.

2. Data Management:

• Build data pipelines to clean, process, and store data efficiently.
• Collaborate with data scientists and analysts to ensure high-quality and readily available data for analysis.

3. Integration and Collaboration:

• Integrate crawlers with existing systems and third-party tools to streamline data access and utilization.
• Work closely with other engineering teams to incorporate new features and maintain compatibility across services.

4. Security and Compliance:

• Implement security measures to protect sensitive data and ensure compliance with data privacy laws.
• Monitor and adapt to changes in social media platforms' APIs, rules, and regulations.

5. Performance Optimization:

• Analyze and enhance the performance of crawlers, reducing latency and increasing data retrieval efficiency.
• Address challenges related to rate limiting and blocking mechanisms.

6. Technical Leadership:

• Mentor junior developers in crawler development techniques and best practices.
• Provide guidance in architectural decisions for data crawling and management.

Key Skills:

1. Technical Skills:

• Strong programming skills in Python, JavaScript, or other relevant languages.
• Hands-on experience with web scraping libraries and tools such as Beautiful Soup, Scrapy, Selenium, or Puppeteer.
• Familiarity with API development and RESTful services.

2. Experience:

• Proven track record in building web crawlers for social media platforms.
• Understanding of platform-specific challenges, including rate limiting and data structures.

3. Data Skills:

• Experience working with big data tools such as Hadoop and Spark.
• Proficiency with SQL/NoSQL databases.

4. Problem-Solving:

• Ability to troubleshoot and resolve complex issues related to web crawling.
• Strong analytical and problem-solving skills.

5. Communication and Leadership:

• Excellent communication skills for effective cross-team collaboration.
• Experience in leading projects or mentoring junior developers.

6. Education:

• Bachelor's degree in Computer Science or a related field, or equivalent experience.
• Advanced degree or specialization in data engineering is a plus.

Benefits Include:

• Competitive salary.
• 100% salary during the probation period.
• Generous paid time off and flexible working arrangements.
• Access to the latest tools and technologies, and the opportunity to work on cutting-edge projects.
• Work-from-home policy.
• Salary bonus, 13th-month salary, and performance review.
• Holiday and birthday gifts.
• Training-on-the-job.