Software and Product Engineering
Model training, Machine Learning pipelines, and product delivery optimization.
PWSLab LFS Version Controlling based on Git, Automated Docker Containerized Deployments
Like most companies serving global enterprise customers, development speed, self-serviceability and the ability to ship fixes and features in a timely manner are the primary reasons for ExpressEarth to pursue new tools. To better meet the needs of their customers, they realized the need to refine their tools to improve speed and agility.
Before rolling out PWSLab, ExpressEarth engineering teams used a combination of more than 70 Subversion repositories, self-hosted Git repositories and some repositories on GitHub. All of these repositories functioned independently, making it difficult for teams to share ideas or take advantage of each other’s code. That’s why a group of engineers partnered with their product development team to standardize on a single, distributed version control tool.
Their team performed an extensive evaluation of tools against a strict set of requirements for Automation in Deployments, Continuous Integration with Containerized delivery. Tools were also evaluated for reliability, functionality like code review, code search, and social collaboration, and their ability to integrate with third-party apps like JIRA and Jenkins.
Although they were migrating a considerable amount of code to PWSLab, it wasn’t a problem to migrate from SVN to PWSLab. Since PWSLab Ops teams have developed an automated approach for SVN to Git migration, it was pretty painless for them to switch to the latest version controlling technology Git.
PWSLab allowed their team to build, train and deploy machine learning models efficiently and at scale on Kubernetes and automation for ML Products allowed them to accelerate the process by which an idea goes from development to production. It helped to achieve several key objectives:
- Fastest time to train, with as much data and as accurately as possible.
- Fastest time to inference, with the ability to rapidly retrain and rebuild.
- Safe and reliable docker (containerized) deployments to observe model behaviour in the real world.
- PWSLab version control of all code, models, and workflows allowed them to concurrently run multiple versions of models to optimize results providing the ability to rollback to previous versions seamlessly. Instead of ad-hoc scripts, the team started using Git push/pull commands to move consistent packages of ML models, data, and code into Dev, Test, Stg and Production environments.
- PWSLab provided central storage and access point for data scientists and software developers looking to work together on machine learning projects, along with an integrated registry to store and manage Docker Container images. PWSLab DevOps streamlined their process of creating machine learning pipelines, data processing pipelines, and integrate AI/ML with existing applications and business processes for their customers.
- Their teams are more collaborative now, discussing and reviewing their code through Pull Requests. It also improved the quality and efficiency of code management. Now through PWSLab they can build, test, and deploy much more quickly and often than before through carefully engineered CI/CD pipelines. All of it is managed mostly through a single interface, PWSLab.
- Centralized infrastructure and internal computation directly saved a few thousands of dollars for ExpressEarth besides indirect savings through product delivery optimization and streamlining.