AWS has recently announced the Amazon Machine Learning service – what is this and what does it mean for customers?
Organizations now hold enormous quantities of data and more data in a wide variety of forms is rapidly being generated. Research has shown that organizations that base their decision making and processes on data are more successful than those that do not. However interpretation and analysis is needed to transform this data into useful information. Data analysis and interpretation is not easy and there are many tools on the market to help to transform raw data into valuable information.
The challenge that most organizations face is that the special skills needed to analyze their data and these skills are not widely available. In addition, to make use of the data the analysis and results need to be tightly integrated with the existing data sources and applications. However, in general, software developers do not have the required data analysis skills. AWS believe that their newly launched Amazon Machine Learning service will overcome these two challenges.
AWS leveraged the data analysis tools and techniques that were developed for the Amazon.com retail organization when designing and building the ML service. These are the underlying tools that try to anticipate the interests of buyers so as to direct them to the item they want and hence to make a purchase more likely. Given the success of Amazon.com these tools and techniques ought to be very useful to the organizations wanting to get closer to their retail customers.
In addition according to AWS, the service can be used without the need for expertise in the area of data analytics. The service provides features that can be used by software developers to build a model based on imperfect data; to validate that the predictions from the model are accurate and then to deploy that model in a way that can easily be integrated without change to existing applications. AWS shared an anecdotal example in which their service was able to create a model in 20 minutes which had the same accuracy as a model that took two software developers a month to create manually.
As you would expect the new service is tightly integrated with AWS data sources such as Amazon S3, Amazon Redshift and Amazon RDS. It can be invoked to provide predictions in real-time; for example, to enable the application to detect fraudulent transactions as they come in
However there are the security and governance aspects of the use of this kind of tool. The recent KuppingerCole Newsletter on Data Analytics discussed the problem of how to draw the line between improving service to customers and invading their privacy. At what point does the aggregation and analysis of data become a threat rather than a benefit? These are difficult questions to answer and regulations and the law provide little help.
However from the point of view of an organization that wants to get closer to its customers, to provide better products, and to become more competitive data analytics are a powerful tool. In the past the limiting factor has been the skills involved in the analysis and machine learning is a way to overcome this limitation.
Using this form of analytics does have some risks. Firstly it is important to be sure of the accuracy of the data. This is especially true if the data comes from a source which is outside of your control. Secondly can you understand the model and conclusions from the analytics process; an explanation would be nice? If not be careful before you bet the farm on the results. Correlations and associations are not cause and effect – make sure the results are valid. Finally are you sure that you have permission to use the data at all and in that way in particular? Privacy rules can limit the use you can make of personal data.
Overall, AWS Machine learning provides an attractive solution to enable an organization to become more data driven. However it is important to set the business objectives for the use of this approach, to define the policies for its governance, and the appetite for risks relating to its use.