Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
Data can drive significant business value for every company. Globally, we are collecting more of it than ever before, and it keeps growing—from 45 zettabytes in 2019 to a projected 175 zettabytes by 2025, according to IDC. However, companies are struggling to monetize all the personal, behavioral, engagement and transactional data from customers and operations and turn it into actionable intelligence to drive their digital strategies. Leveraging insights from your data is critical for success and the development of new digital solutions to better serve customers, grow sales, improve operations and more.
Data helps organizations develop the right software for the right customer at the right time. Data-driven organizations are 23 times more likely to acquire and retain customers than their peers, and are growing at an average of more than 30% annually, according to Forrester.
Organizations with a deep and dedicated focus on data and analytics tend to follow three common paths to successfully drive their software development initiatives.
Discovery Matters
It is nearly impossible to draw value, understanding, and insights from the massive amount of data businesses collect without a proper discovery system in place. Having a comprehensive process of collecting data from various databases and silos and consolidating it into a single source that can be easily and instantly evaluated is critical for any software development project.
During client engagements, data scientists perform deep-dive research into clients’ data pools and workflows to support client roadmaps. We recently partnered with a global e-commerce company to create a recommendation system for their customers. The discovery process allowed us to verify the feasibility of recommended products using their existing data before moving ahead with the project. Using machine learning, the team was able to clean and analyze information from multiple angles. With that knowledge and confidence, the rest of the development project proceeded quickly knowing the data was in place and the project was viable.
Align KPIs
Software development metrics help organizations and their partners track and measure how an application is performing and how effective the development team is in its work. IT organizations rely on a range of KPIs to fully understand software engineers’ progress, as well as gauge software quality including performance, engagement and user satisfaction. While each engagement is different and requires a different set of KPIs, ensuring that the organization and development team are on the same page is crucial.
During the e-commerce project, success was measured through conversion and user retention rates. Other KPIs that are regularly used include developer productivity and performance metrics like cycle time and team velocity, and software developer metrics like response time, usability and UX data, among others.
Deploy Agile
By using agile methodologies at all stages of development, developers are able to focus on business results, not just architecture and new technologies. A similar agile approach can also be applied to data management. When it comes to data science, it’s all about extracting useful information from raw, structured and unstructured data and implementing machine learning models to analyze vast amounts of information. This process requires more creativity and is a key driver for understanding potential risks and failures over the course of any given project.
With agile, data models and analyses are broken down into manageable bite-sized chunks that can be built and tested quickly—the same way software is developed. If the analytics do not yield the expected results, data scientists see the failures immediately and can correct their course. They could ask the team to analyze different data sets, adjust the models, tweak algorithms or make an educated decision about the project’s viability.
A multinational credit company took this approach to the next level when it used agile to develop a new, big data hybrid cloud platform. After anonymizing, aggregating and storing information in a data lake, the team quickly realized the database it was planning to use was limited to a certain number of partitions and entities. By running agile sprints and developing and testing the minimum viable product (MVP) continuously, the team was able to seamlessly switch to a new database that had more scale and functionality.
The value of data is only going to increase with time. Data-driven organizations embrace the use of data and analytics to gain unique business insights to support their software development projects and overall business. By focusing on the data in each and every development project, companies have a tremendous opportunity to gain a competitive advantage and grow their businesses.
