Enterprise Databases in the Cloud
This Leadership Compass provides an overview of the enterprise databases offered primarily as managed services to enable modern cloud-native application architectures as well as data processing and analytics at scale. We examine both the portfolios of large cloud service providers and innovative modern managed database offerings to provide you the guidance and recommendations for finding the right technology for your next application or a cloud migration project.
As businesses continue embracing the cloud to achieve better agility and innovation, lower the time to market for their digital services, and eliminate the costs of maintaining their own infrastructures, more and more corporate data migrates to the cloud as well. These developments do not just affect traditional enterprise workloads but cover a broad range of new use cases like IoT telemetry collection, business intelligence, or security analytics.
Market trends indicate that companies are increasingly developing cloud-first strategies for their data management, even when facing additional compliance regulations. Scalability, flexibility, range of services and predictable costs – these are the primary factors for the growing adoption of managed database services in the cloud.
Nowadays, most sufficiently complex cloud migration projects or even cloud-native applications inevitably end up using multiple types of databases and other data stores for structured and unstructured information depending on their business requirements. Recently introduced data protection regulations like the European Union’s GDPR or California’s CCPA make no distinction between relational databases, data lakes, or file stores – all data is equally sensitive regardless of the underlying technology stack.
Thus, in this KuppingerCole Leadership Compass, we are focusing primarily not on the technical capabilities of individual database management solutions that can be optionally deployed on a cloud infrastructure. Rather, we want to focus on vendors that offer comprehensive business-focused solution portfolios that cover a broad range of use cases, whether this functional scope is supported by a single universal database engine or a vendor offers a choice of individual products to choose from. We also consider solutions outside of the traditional definition of a database management system, provided that they address a relevant business requirement for storing, managing, and processing data in the cloud.
Here are some of the solutions that we consider relevant for this Leadership Compass:
- Traditional enterprise DBMS offered as managed services in the cloud
- Cloud-based Big Data platforms for large-scale analytic workloads
- Vendors or service providers offering comprehensive suites of independent database engines for various usage scenarios: relational, document, graph, key-value, and other types under unified management
- In-memory databases and computing platforms or similar technologies
- Other innovative data management products with a strong cloud-first focus.
1.1 Market Segment
Databases are arguably still the most widespread technology for storing and managing business-critical digital information. Sensitive financial transactions, confidential customer records, sensor data from IoT devices, or just some other kind of business-relevant information – whether on-premises or in the cloud, they end up stored in a database for management, archival, analytics, or any other kind of processing. However, with the growing adoption of modern, cloud-native application architectures and simply because of the increased volumes and complexity of information that ends up being stored in the cloud, enterprise applications can no longer rely on a single type of database to address the broad range of requirements. For example, developers might opt for a specialized time-series database to record sensor data from IoT devices, store collected information in a data lake to simplify analytics but rely on a distributed NoSQL database engine to power their client application.
In fact, it would not be an overstatement to say that the very definition of “database” for many organizations is radically changing. Nowadays, a cloud-native enterprise IT project simply cannot stick to traditional monolithic designs. Instead, loosely coupled distributed architectures like microservices are preferred. Applications that are deployed across multiple cloud regions or even multiple clouds are a norm. These modern architectures introduce new requirements for database management systems.
Developers expect databases to be cloud-native as well: able to support highly distributed and scalable deployments, withstand planned and unplanned outages, accommodate multiple different data models while still allowing to access analyze data across them with low latency, and, last but not least, implement reliable and compliant data protection and threat mitigation controls. Ideally, a modern enterprise database should be completely managed by a third party, allowing customers to focus on application logic instead of infrastructure operations and maintenance. In a way, an ideal enterprise cloud-native database is the one that does not expose any of its underlying infrastructure: a serverless database for serverless computing.
Unfortunately, such a dream database solution that would universally fit all potential cloud-native use cases probably does not exist yet. Still, a market for enterprise databases in the cloud is quite substantial and will definitely continue to grow and evolve, since the demand for such solutions is immense. Whether a particular vendor can address such a broad range of business requirements with a single integrated service or provides a selection of specialized services in a suite is not the primary focus of our analysis: instead, we are looking at the overall ability of a vendor to support a potential customer in as many use cases as possible.
In this report, these are the key areas we were looking for:
- Solutions that provide enterprise-level data management capabilities for various usage scenarios
- Solutions which are offered as managed services within a single or multiple cloud environments
- Solutions that take advantage of the core capabilities of cloud services: scalability and performance, managed or automated patching, tuning, and optimization, etc.
- Solutions that implement the necessary data protection and compliance controls to prevent data breaches and violations of government or industry regulations
- Solutions that provide comprehensive centralized management, monitoring and analytics capabilities integrated with other cloud services, vendor’s own or third-party ones.
This Leadership Compass is one of the first KuppingerCole’s reports investigating this approach towards market analysis. This means that several notable vendors were unable to participate in our rating for various reasons – they will still be mentioned in the later chapter “Vendors to watch”. Still, even the relatively small selection of companies we do have in the main rating demonstrates a broad variety of approaches companies are utilizing to address the customer requirements for a truly universal cloud database service.
We have a mix of traditional database vendors like Oracle, which work hard to elevate their decades-long database expertise to the cloud scale; veteran cloud service providers like AWS that strive to offer their customers a broad selection of specialized database engines; and, of course, young innovative companies like MongoDB that had to start from scratch to design their products to be truly cloud- and even multi-cloud-native. We even have a couple of vendors that don’t offer their own database services but help make database operations and data management in the cloud more efficient and secure.
1.2 Delivery models
Since this report focuses on enterprise databases in the cloud, we are explicitly excluding products that are primarily delivered for on-premises use and only support optional IaaS-based cloud deployment. For example, we would not consider “vanilla” PostgreSQL for inclusion, yet quite a few vendors do offer cloud-based managed database services implemented using PostgreSQL technology.
We are also not including solutions that only address a small subset of database management capabilities (for example, only a key-value database service) unless they are offered as a part of a broader portfolio. Finally, we are not covering additional tools and capabilities such as data migration or analytics, unless they are provided as an integral part of a data management service.
Thus, native support for cloud deployments is the primary criterion for inclusion in this Leadership compass; support for multi-cloud and hybrid deployment scenarios is also considered a major advantage. Also, here are some features and criteria that directly and positively influence a solution’s rating in our evaluation:
- Integrated protection controls for data integrity and confidentiality (including but not limited to encryption and masking, segregation of duty, and user behavior monitoring). Please note that KuppingerCole has a separate Leadership Compass devoted entirely to the topic of Database and Big Data security. Still, data protection and compliance features are one of the required capabilities for this rating as well.
- Minimized downtime for maintenance, as well as resilience against unplanned downtime. This is one of the crucial yet often overlooked criteria for selecting a cloud database capable of providing the highest availability for business-critical applications: remember that planned downtime is often not included in a service level agreement.
- Compatibility with existing on-premises database management solutions, as well as other means to simplify migration of existing data and workloads to the cloud. By relying on a consistent set of supported APIs and protocols across on-premises and cloud databases, application developers can not only dramatically reduce the time to market for their apps, but also minimize the danger of vendor lock-in when using proprietary interfaces.
1.3 Required Capabilities
When evaluating the products, besides looking at the aspects of
- overall functionality
- size of the company
- number of customers
- number of developers
- partner ecosystem
- licensing models
- platform support
we also considered the following key functional areas of database platforms in the cloud:
- Supported Data Models – we expect an enterprise-class cloud database service to support multiple data models beyond the traditional relational databases. These might include multiple types of “NoSQL” data models like key-value, graph, or document databases as well as more specialized database types like time-series databases or even centralized and distributed ledgers.
- Integrated Management – whether a vendor offers a universal multi-model database engine or many specialized database services, we still expect it to provide unified management and monitoring, as well as a consistent approach towards relieving the customers from managing the underlying infrastructure.
- Database and Data Security – an enterprise-class database service must implement a comprehensive set of data protection capabilities, fine-grained access controls, activity monitoring, audit, and compliance features as well as offer comprehensive multi-layered protection against external and internal threats.
- Data Management and ETL – here we evaluate the capabilities and services offered by the vendor to enable migration of existing data and applications to the cloud, integration with external data sources, as well as managed extract, transform and load capabilities.
- Business Analytics – although we do not expect every cloud database service to offer its own business analytics solutions, making the data in the cloud available for 3rd party analytics tools at scale is a crucial capability. This covers such functionality as managed data warehouses and data lakes, as well as optional AI/ML technology integrations.
- IoT and Real-Time Processing – capturing streams of data in real-time is an increasingly popular use case for cloud applications, especially relevant for large fleets of IoT sensors, but also financial transactions, event streams, or security intelligence. Support for reliable and secure data streaming with real-time analytics is another crucial enterprise cloud database capability.
- DevOps and Integration – this includes providing a broad set of tools and integrations for developers to support CI/CD pipelines, enabling full information lifecycle management including proper disposal of test data as well as supporting DataOps methodology with technologies like data virtualization and self-service functions for developers, data scientists, and other data consumers.
- Deployment and Scalability – while not a functional capability per se, it is a critical requirement for all cloud-native database services to be able to withstand high loads, support scaling without any significant limitations and support deployments in high availability configurations. Databases specifically designed with modern highly distributed application architectures in mind will receive high ratings here as well.