Career Profile
Hello There!
Bill Here! I have been working since 2008 exclusively with DATA - 16 years and counting.
I’m a Data Engineer & Data Architect that focus on building scalable, reliable, and world class’s Data Solutions.
I have a huge experience in Data Driven Development, Data Integrations, Data Modeling, Data Engineering, Data Architecture, Data Platform and AWS Data Services.
My main focus is Data Engineering, Data Architecture ( specially Data Mesh ) and Cloud Computing using all of this to build Scalable & Cost Effective Data Solutions.
Talk Data with me, and let’s change the world using DATA!
Experiences
Responsible for Data Platform used in the whole Data Analytics Team at BTG & Data Mesh Expansion adding more Data Producers and Data Consumers.
Some technology of our Data Platform which I’m responsible:
- AWS Technology Stack: Redshift (RA3, Serverless), Athena, EMR, ECS, Fargate, Lambda, StepFunctions, KMS, Lake Formation, Glue Data Catalog, EKS.
- Kafka: Usage of Amazon Managed Streaming for Apache Kafka (MSK).
- Airflow.
- DataHub for Data Catalog.
I also work closely with Data stakeholders in order to build and sustain a Self-service Data Platform following the Data Mesh Principle.
Focusing on enable a cost effective Data Platform to ensure the best results for the business as well for all stakeholders.
Design, build and implement Data world class solution using AWS technology, focusing on Data stack, using services like Redshift, Kinesis, Fargate, Aurora, EMR, Lake Formation, and others.
My focus is to help the Data Analytics team to build the best Cloud Data Platform and implement Data Platform Foundations.
Work closely with business unit to build, implement and monitoring Big Data pipelines.
Build and maintain Data solutions using AWS technology:
- S3
- EMR
- Kinesis
- Redshift
- Others AWS Data Technology
Main Projects:
-
Migrate Data Lake to AWS Lake Formation
I migrate the Data Lake to be entire managed by Lake Formation, Data Analytics team needs to share data cross account using the Lake Formation feature, to enable this feature the Data Lake needs to be managed by Lake Formation instead of IAM.
Tech stack: AWS Lake Formation, AWS Redshift, boto3.
-
Income model MLOps
This was my first MLOps project, the Data Science team build the model and I help them to deploy this model in production, to achieve this challenge, I architect the whole solution using AWS ECS Fargate, Kinesis Data Stream, Kinesis Firehose and DynamoDB for model metadata. I architect a Serverless solution.
Tech Stack: AWS ECS Fargate, AWS Kinesis Data Stream, AWS Kinesis Firehose, AWS DynamoDB, AWS Athena, AWS Glue data catalog, AWS Lambda, parquet.
-
Legacy DW migration to AWS
Migration the legacy DW solution to AWS, to achieve this I built a Cloud native solution using Amazon MQ to receive the statement data, AWS Fargate to write the received data in AWS Aurora (Postgre engine), move the OLTP data from Aurora to Redshift using Federated Query, and orchestrated the batch job using AWS Step Functions.
Least but not last I used scheduled query to unload the data from Redshift to S3 in parquet format, enabling Data Science team could analyse the DW data.
Tech Stack:
- AWS Amazon MQ (RabbitMQ)
- AWS Redshift
- AWS Step Functions
- AWS Glue Data Catalog
- AWS Federated query
- AWS Aurora (Postgres engine)
- AWS Redshift Scheduled queries
- AWS ECS Fargate
- AWS Lambda
Proxify is an award-winning Swedish tech company matching the world’s best tech talent with leading companies.
Proxify embraces a people-first approach, with the goal to transform futures for companies, and talented professionals alike.
At Proxify, we live and breathe our values.
- Speed: We seize opportunities and embrace our mistakes as a key ingredient of innovation.
- Impact: We’re driven to create positive change together, fuelled by an uncompromising dedication to our clients.
- Quality: We’re always meticulous and dedicated to data. We celebrate excellence everyday.
- Personal: Empathy is our north star. We believe that anything is possible, with a little help from our friends.
More details: https://proxify.io/company
Data Engineering freelance projects: Big Data, Data Lake, Cloud Computing using AWS Stack.
Key technology’s used:
- AWS
- AWS ECR
- AWS ECS
- Terraform
- Airflow
- AWS Lambda
- DynamoDB
- AWS EMR
- AWS Athena
- Build and maintain data ingestion pipelines using AWS stack and Airflow.
- Develop data modeling algorithms.
- Manipulate and analyze complex, high-volume, high-dimensionality data from varying sources.
- Monitoring performance and advising any necessary infrastructure changes.
- Collaborate with partners and development team to drive analytic projects end to end.
- Data Visualization Server Admin (Tableau Server).
- AWS Redshift Admin.
- ETL development for main company system such as MES (Manufacturing Exection Systems) and SAP.
- Massive use of Pentaho Data Integration, development ETL process for all legacy system that used PL/SQL as ETL.
- Data driven application development for Production Planning, Aircraft Delivery Center and International Logistics.
- Multidimensial database modeling for BI tools.
- Production planning data cleansing in Boeing carve-out process, a lot of data to analyze and clean, a huge data cleansing processing.
Software development and process automation for Production Planning team.
Main Technologies: .NET, PL/SQL, Oracle, OLAP, JavaScript, AngularJS, CSS.
Certifications
Earners of this certification have an in-depth understanding of how to use AWS services for data collection, storage, processing, and visualization. They demonstrated the ability to leverage AWS analytics tools for deriving business value from data. Badge owners are able to leverage various AWS services to manipulate collections of data for organizations of all sizes.
Earners of this certification have an in-depth understanding of how to compare AWS database engines with one another to know when which one should be used. They demonstrated the ability to leverage both relational and nonrelational engines. Badge owners have technical expertise to select optimal engines and design solutions to improve performance, reduce costs, and enable innovation for organizations of all sizes.
Earners of this certification have a comprehensive understanding of AWS services and technologies. They demonstrated the ability to build secure and robust solutions using architectural design principles based on customer requirements. Badge owners are able to strategically design well-architected distributed systems that are scalable, resilient, efficient, and fault-tolerant.
AWARDS
The AWS Community Builders program offers technical resources, mentorship, and networking opportunities to AWS enthusiasts and emerging thought leaders who are passionate about sharing knowledge and connecting with the technical community. My cohort is Data and during the program I participate in technical talks about data architecture; data engineering; learn new AWS technology with AWS experts; produce and share technical contents; help to engage the AWS Community; Keep learning new knowledge about Data world (Lifelong Learner).