A while ago I entered the challenging world of Big Data. As an engineer, at first, I was not so impressed with this field. As time went by, I realised more and more, The technological challenges in this area are too great to master by one person. Just take a look at the picture in this article, It only covers a small fraction of the technologies in the Big Data industry…
Consequently, I created a meetup detailing all the challenges of Big Data. Especially in the ecosystem of AWS cloud. I am using AWS infrastructure to answer the basic questions of anyone starting their way in the big data world.
How to transform data (TXT, CSV, TSV, JSON) into Parquet, ORC? which technologies? using SQL or code?
Which technology should we use to model the data ? EMR? Athena? Redshift? Spectrum? Glue? Spark? SparkSQL?
How to handle streaming?
How to manage costs?
Performance tips?
Security tip and network tips?
Cloud best practices tips?
The first lecture video, spoken language — english:
https://youtu.be/hZHaaUBKxAI
Slides:
https://www.slideshare.net/OmidVahdaty/aws-bigdatademystified-11-big-data-architecture-lessons-learned-english

Hi, I am Kiat, a pre Data Engineer and present Full Stack Developer in Malaysia.
I am having a full time job - Full Stack Developer under Razer Pay.
I am able to take 1 big project only per time. Please contact me any anytime.
Services available:
1. Develop website and web apps
2. Develop mobile apps
3. Big data processing and management
Clients:
1. Government
2. Yoodo
3. AeonBig
4. TheStar
5. Nike
6. KFC
7. Razor Pay

With over 15 years developing knowledge management systems, data management, web scraping , and web applications on Python, plus over 5 years experience on Big Data technologies on Apache Hadoop & Spark, I can help your organization get value from your data rapidly and fast.
Education: Bachelor of Information Technology (Hons), Major in Software Engineering, Minor Digital Electronics. PETRONAS University of Technology.

An enthusiastic and driven student of Business and Commerce with a major in Business Analytics. Willing to learn new concepts quickly and seek the opportunity to gain experience. Analytical, purposeful student eager to utilize skills and knowledge gained through education and experience to assist organizations to drive business growth. Exceptional communication skills and strong ability to lead teams through demanding situations. Proficient technical knowledge including Microsoft Excel, Microsoft Word, Microsoft PowerPoint, SAP Logon, SAP Lumira Discovery, SAP Analytics Cloud, WEKA 3.8, IBM SPSS, RStudio and EViews.

A college student, Having Internship experience for 5 month

Collaborate with multiple teams and build Spark and Storage clusters.
Build, maintain and deploy data science frameworks
Automate, and Monitor ETL pipelines and ML workflows

I am Rashid Sarfaraz. I have a bachelors computer Science (Software Engineering) and a masters in Computer Science (software engineering) . I would like to call myself a technologist, I am excited to learn, Implement and teach new technologies..
I Worked for over 14+ years on building enterprise applications, reporting analytics ,matrices / KPIs development , Specially SAP Business One, Hana DB .
Well versed in Test Planning, Test Case Design, Defect management & Test case documentation..
Automation Tools: Selenium RC, Selenium Webdriver, Selenium Grid, Selenium IDE and AutoIT.
Build Tools: ANT, Maven, and Jenkins.
Involved in design and development of keyword-driven automation framework using Selenium Webdriver.
I have consulted with large organizations in Government, Telecom, Power, FMCG sectors and start-ups. I manage projects using agile methods and tools. I have been involved in for the last 7 years corporate training including SAP, Oracle, Cost Accounting, Purchasing, Production, Supply-chain and Business Analytics. I have around 150 corporate batches trained. My clients include Consulting Companies, Implementation partners, Consultants and other professional students etc....

➢ Data Architect with 12+ year of experience. Experience in Architecting and implementing Big Data & Analytics solutions, BI/DW interfaces & Projects in Data Analytics with a focus on Banking, Telecom, Life Science & Logistics domain, while exceeding expectations on client satisfaction and timely project delivery.
➢ Architect and Implementation of Big Data Solutions using Hive, Sqoop, Spark, Scala & Kafka.
➢ Proficient in Machine Learning Algorithms and creating various models for ML using R & Python.
➢ Expertise in BI & DW Solutions using IBM DataStage, SQL Queries, Unix Shell Scripting & Tableau.
➢ Experience of Requirement Gathering, Data Modelling & Data Warehouse.
➢ Experience in creating Data Models using Toad Data Modeler.
➢ Knowledge of Google cloud and snowflakes cloud platform.

I have 5+ years of experience in data science. I can help you with most of the skills which is required for solving data science problems.
Together we can work with Big data Spark, Kafka,Flume, Hive, Impala, Oozie, Airflow, Azure, Google Apigee and Streaming and batch data etc.

Data Engineer helping organizations to achieve true data potential
Microsoft certified solutions expert with more than 10 certificates in data & AI stack
Hands-on experience on data processing, data wrangling, data governance, data visualization with more than 10 years of professional experience across industry verticals
Passionate about data science and new data technologies
Customer focused, hardworking, innovative, fast-learner, proactive and self-motivated
Technical Languages: SQL | Python | Scala
Data Processing tools: Spark | Azure Cosmos DB | Azure SQL Data warehouse | Azure Data factory | Event hubs | Databricks | Kafka | SQL Server | SSIS |
Public Cloud: Azure