Senior big data developer - Sphinx WorldBiz LimitedPosted by Sonali Mishra on October 18th, 2019 The Big Data Developer builds what the big data solutions architect has designed. Big data developers develop, maintain, test and evaluate big data solutions within organisations. Most of the time they are also involved in the design of big data solutions, because of the experience they have with Hadoop based technologies such as MapReduce, Hive MongoDB or Cassandra.
He builds large-scale data processing systems, is an expert in data warehousing solutions and should be able to work with the latest (NoSQL) database technologies.
A big data engineer understands how to apply technologies to solve big data problems and to develop innovative big data solutions. In order to be able to do this, the big data engineer should have extensive knowledge in different programming or scripting languages like Java, Linux, C++, PHP, Ruby, Phyton and/or R. Also expert knowledge should be present regarding different (NoSQL or RDBMS) databases such as MongoDB or Redis. Building data processing systems with Hadoop and Hive using Java or Python should be common knowledge to the big data engineer. A big data engineer generally works on implementing complex big data projects with a focus on collecting, parsing, managing, analyzing and visualizing large sets of data to turn information into insights using multiple platforms. He or she should be able to decide on the needed hardware and software design needs and act according to the decisions. The big data engineer should be able to develop prototypes and proof of concepts for the selected solutions. Like it? Share it! More by this author |