Data Analytics - The Birthplace of AI and ML
May 10, 2021 by Team SetConnect
Growth in usage of Data Analytics
Data analytics continues to grow rapidly fueled by the enormous amounts of data generated through the growth of social networks. In addition, the acceleration of e-commerce transactions and the increasing number of IoT devices and sensors have substantially contributed to the amount of data being captured. According to growth projections from Statista, 74 zettabytes of data will be created in 2021 as compared to 59 zettabytes in 2020 and 41 zettabytes in 2019. A zettabyte is a trillion GB !
In organizations, there is a larger movement towards adoption of analytics. The usage of descriptive analytics, while limited in value has increased, especially because of data visualization. End-users now, are directly able to manipulate data using popular enterprise software. The more complex, yet important aspects of analytics, namely predictive analytics has also been growing in adoption. Several predictive modeling applications in marketing, finance and supply chain amongst others are replacing the older practice of relying merely on descriptive models. The global market in predictive analytics is expected to grow by more than 3 times between 2020 and 2025 (Source: marketsandmarkets.com).
Challenges in data analytics
In spite of this impressive growth, implementation of data analytics projects continue to experience challenges. Not all of the activities and decisions that need to be taken by organizations fall within the scope of current modeling techniques. With data gathered through social networks increasing coming under regulatory lenses, privacy and data protection need to be addressed. For many predictive models, the amounts of data required to build meaningful and accurate models is huge and data collection is often daunting. Lack of technical knowledge and insufficient talent and experience in organizations also is a significant deterrent to project success and value creation. Building complex models is another area where many struggle. In addition, analytics solutions are often point solutions – ie; they address narrow areas of the organization or business function even though the business problem may span across multiple functions.
Some companies find it difficult to justify analytics projects for want of adequate business cases. Lack of experience in implementing successful projects and inability to take the initial risks also act as hurdles to getting top management support. Many businesses are still to make significant headway in their digital transformation efforts. With revenue growth affected by the pandemic, several organizations are adopting a wait-and-watch approach when it comes to take new initiatives on-board. Companies who have benefited from analytics applications on the other hand are now rushing to acquire expertise in these areas to improve and maintain their competitive edge. However, this huge demand for expertise is also impacted by talent shortages.
While some of the challenges related to the Covid-induced lack of growth and ability to invest will be overcome in time, the key hurdles – managing large volumes of data, availability of necessary talent, expertise and experience and complexity in building models remain. What then are some of the potential solutions to tackle these issues and usher in more analytics projects and indeed an analytics mindset across the institution?
AI / ML and Analytics
Artificial intelligence (AI), simply put is where a machine seems human-like and can imitate human behavior. Some of these behaviors include planning, problem-solving and learning. How is this achieved? By analyzing large amounts of data and pattern recognition, AI systems follow rules to manage responses to specific patterns. Applied AI – in which systems are designed to address specific applications (such as factory robots or trading in stocks) have been growing in popularity.
Machine learning (ML), is a sub-set of artificial intelligence. ML is primarily used to process large quantities of data very quickly using algorithms. These algorithms have the ability to change over time and improve in their performance. A factory might collect data from machines and IoT sensors in large volumes, often much more than what any human being is capable of processing. By continuously monitoring and analyzing the data, ML algorithms may identify exceptions or anomalies. Specific actions may then be used to address these exceptions (often by people).
Some of the areas where AI/ML has found success include autonomous vehicles, speech recognition, healthcare informatics, computer vision and database mining. Supervised machine learning techniques include parametric/non-parametric algorithms, neural networks and support vector machines. Unsupervised learning methods include clustering, dimensionality reduction, recommendation engines and deep learning. AI/ML applications are growing beyond silo applications and are now helping businesses in streamlining processes. They often uncover new elements of data and situations which help in improving decision making.
How can AI/ML help data analytics?
With advanced work being done in AI/ML, especially in the area of automation, these could address some of the challenges that exist in current implementations of data analytics. New tools and application software solutions are emerging and these could address some of the challenges identified. Some of these solutions provide end-to-end automation of thekey processes involved in delivering an analytics solution. They also provide support to some level of decision making so that key insights and recommendations are provided directly to decision makers without going through the intervention of analysts.
These analytics process automation tolls focus on achieving targeted business outcomes. They straddle across existing enterprise systems such as ERP and CRP as well as discrete data sources. Starting from data preparation and improving the quality of data the software tools provide support for in-depth visualization and combining traditional data with support of maps and logistics information. The applications provide guidance for building complex models with a step-by-step assistance. They also support the construction of complex models with little or no coding requirement.
The other capabilities that AI/ML tools bring is the incorporation of models that span across business functions. As an example, in a supply chain application, the organization ERP system contain information related to product lead times and inventory levels. Connected with a forecasting analytics model, as well as transportation capacities this could be used to predict the demand fulfillment requirements and consequently optimize inventory levels. The transportation requirement could then communicate with the HR sub-system and assist in managing optimum crewing requirements for product distribution. This of course would require detailed knowledge of the various stages of manufacturing and distribution across the supply chain as well as the HR elements. Such a solution will be significantly more powerful and valuable than having individual models for forecasting, transportation and inventory planning.
Unlike earlier stand-alone models, these AI/ML solutions will not require the user to interact with each model, take decisions and interface with the next model. These could create a workflow across models. In addition to automatically finding the right pieces of data and connecting, the different aspects of the analytical process will increasingly become automated and supported by AI. Not only will this quicken the deployment of solutions, but will also require fewer people to manage the analytics functions. Potentially these cross-functional applications will be more valuable then their siloed counterparts. In summary, AI and traditional analytics will combine to make data analytics easier, more effective and more valuable to companies.