Ingestion engine with PySpark

I worked on a project in which it was necessary to perform multiple ingests of information to the datalake (on-premise). Up to that moment, the client performed new ingests of information as 100% new developments, implementing validations and processing established by the users. Performing ingests in this way mainly generated the following problems Repeated and not very scalable code. Repeated validations and processing, in case of a change it impacted on all the processes already developed....

October 23, 2022 · 1 min · Robert Huaman


All companies/customers have an established change flow for deploying components to the production environment, some more complicated and bureaucratic than others. Where I work, the change flow to deploy to production was a very tedious and manual process. Although there were automated pipelines in jenkins to deploy components to production, in the Jira ticket the components to be deployed had to be documented, for auditing, security and to ensure a correct rollback in case the deployment to production failed....

October 23, 2022 · 2 min · Robert Huaman

Legacy Checker

Legacy Checker, is a tool that is injected into the Remedy front end (HTML), in order to perform validations on the tickets and thus avoid making mistakes, with which the following benefits were achieved Early identification of errors in the ticket. Reduction of ticket deployment time (since for each error in the ticket, the initial state is reverted to the initial state). Reduction of manual errors. Languages/Technologies: Javascript (ES6) Regex Gulp OOP

October 23, 2022 · 1 min · Robert Huaman