Analyst Chat #80: AI Service Clouds

Welcome to the KuppingerCole analyst chat. I'm your host. My name is Matthias Reinwarth, I'm lead advisor and senior analyst with KuppingerCole analysts. My guest today is Annie Bailey. She's an analyst working with KuppingerCole on emerging technologies. Hi. Hi Mathias. Thanks for having me back. Have You, and the first time with video. So we are improving here as well. So you've brought an interesting topic with you and you, I think this is really an important thing to start with.

Um, everybody currently is talking about emerging technologies, of course, but as part of that, many organizations are looking into machine learning and this is far from theory. This is really something many organizations are now looking at to leverage these technologies, these machine learning technologies for their individual business, for improving business processes and for creating really new business models.

Um, the question is, um, since this is an emerging technology, uh, and we see this skill gap in many areas of it, I guess that's the same with machine learning as well. There are not that many trained machine learning, engineers and technicians, and really creative people being capable of using this machine learning technology in a proper way.

Um, how can enterprises still use this technology? Are there ways of dealing with that without having to hire experts? Exactly.

And so, as you can imagine from other areas of technology where there are fewer people who have the necessary skills to put in tools into implementation, that's exactly the same for AI and machine learning. There are people out there who have the knowledge and the skills, but not nearly as many as would be desired.

Um, and so that leads to an interesting implementation question. How on earth are enterprises going to get their hands on the technologies with the right amount of knowledge, um, to use something like AI and machine learning. So they're, first of all, has to be some backup for those experts who are out there since there's not enough people to do the work, which is being required.

Um, there has to be some level of support. Um, so that projects, uh, don't require as many, um, machine learning developers and engineers.

Um, and so there's a quicker time to value. So we're looking for this support aspect, but there's another really important aspect which comes into play here, which is the diversity of teams and taking a multi-stakeholder approach towards using AI and machine learning for, um, public facing products and services to make sure that every type of person, regardless of age of gender, of ethnic background, um, all of these, um, sensitive areas, we have to make sure that people, um, are, uh, that their needs are met equally.

And that accessibility is still brought through even when decisions are being made with support from AI and machine learning. So to, uh, come back to our topic, we need to have a multi-stakeholder team.

Um, and that requires people who of course, are experts in AI and machine learning, but also people who are from different fields. So for example, if you're implementing a project which will go into youth in a hospital, then you should consider the end user. So potentially the doctors, the nurses, the intake staff, um, who would be eventually using this model, they should have a say and be able to give their, um, frontline input into the project out of it's being developed so that it better meets the needs.

Um, so this requires an approach which brings in many people of many different expertise. Right? Exactly. And then the question is really when we have these diverse teams, so we, on the one hand have the, the, the subject matter expert that domain know how w within the organization, which is not necessarily AIML related, but it can be leveraged or needs to be leveraged. And on the other hand, there are these technology experts that there are these machine learning experts that can transfer this domain.

Know-how into something that is an, a machine learning model, a machine learning approach towards the solution, but still there is some requirement for, for assistance, for support for, for service. How could this look like? So exactly. So to bring in, uh, these needs of supporting the developers who are on the project and to bring in, uh, different types of people with different types of expertise, there's a product which is called AI service cloud, or a category of products rather.

And this is to bring detailed developer control to the AI and machine learning development process and to these experts, but it also abstracts away some of the complexity so that different subject matter experts can weigh in. Okay. But That sounds like a really a new type offering a new type of service, a product.

Um, and instead it's an AI service cloud. Um, how can I imagine, how should I, should I think of something like that, how does that look like in reality? So it's campuses quite a few different things.

Um, and so it's really more than just enabling different types of people to work on the same AI project. It typically breaks down the model development life cycle into its main phases. So the process of actually building a machine learning model, um, and then it typically guides, or it automates, or it gives full developer control over each one of these phases.

Um, and of course then for training and for implementation, this requires, uh, quite a lot of computing power. Um, and so these AI service clouds typically provide the computing power needed to implement or to train these models or to retrain later on in its life cycle.

Um, at some of these specific, um, needs for governance, um, including model explainability, um, and also bias mitigation. They typically offer modules that address those two areas.

Um, and then on top of all of this, on top of this toolbox for designing and implementing your own model, you typically have access to pre-trained or kind of plug and play AI solutions for things like chat bots for translation, for image classification, for these, um, very, um, very general AI use cases, um, which many people need. So they come as a pre-trained option, which you could then customize to your needs. Okay. That sounds like an, an easy entree into a machine learning usage for organizations who are otherwise not yet well prepared to achieve that.

So it's really something that, that offers, um, a nice segue for, for non ML savvy people into this area. Um, and as I am not an expert as well, you've mentioned these phases that they, um, that these AI service clouds guide organizations through these phases, but how should I think of these phases? What are the main phases in this model development for machine? So you typically start with, um, data preparation.

And this actually is most of the work where people typically spend quite a lot of time because you're, first of all, determining what data you have, which could be used to train a model, but also, which would be able to, um, would be your input for the model once it's an implementation.

So this would in many cases must be labeled, um, must be cleaned, must be tested to make sure that there's, um, that, that it is indeed a representative representative sample of the data and the case that you would like to be assessing, but also that it doesn't skew, um, the results towards, um, any one decision or the other. In other words, that it's not biased towards your own business practices, um, known or unknown, um, or on a mathematical level as well, that it is indeed, um, a representative sample.

Um, so that's the first stage data prep. Um, then you would go into model selection. So now there are tons of different algorithms and models to choose from which range from different types of machine learning.

Now, these are sometimes proprietary, which would come with the AI service cloud, um, but a lot are open source. And so determining a model, which is, uh, the most accurate, which delivers the type of results, um, that you would like to see if there's a focus on accuracy, or if there's, um, a focus on, uh, timeliness or different, um, different models of course, can be specialized towards different KPIs.

Um, this is exhausting and time consuming to do a thorough search of each of these models and determine which one is the best. So this is often automated, um, the terms which has chosen this typically auto ML, um, which then applies machine learning to select the, um, model, which would be most useful in your own machine learning case. So it's a bit of inception here, um, going on once the model has been selected, then you need to go through training and validation of that model.

So, um, uh, using your own data, which you've already prepped, um, to train the model to your own use case. Uh, and then of course, to validate that model, to make sure it is delivering reliable results, results, which are useful, um, and accurate, then there's implementation ongoing monitoring to again, make sure that there's no model drift that your data and also your model, um, stays consistent over time and continues to measure what you want it to measure.

Um, and to be aware if the types of data inputs, which are coming in have slightly changed, which would mean the results would slightly change as well. Um, and then there's retirement to make sure that the model and also its data is, uh, put to rest when it needs to be.

Um, and so if we take all of these phases together, um, this is typically called an L ops or machine learning operations. Okay. Of course this term is nicely coined I'm related to dev ops in that area, of course know about this, but this time looking into machine learning.

And I think that's also quite an interesting aspect to look at as well, because when we say we're going to talk about, um, AI service clouds, there's this term service in there, and service means that this is something that is provided as not a one size fits all, but, but a selected range of capabilities of services that are combined to fulfill a range, um, of, of tasks to, to solve different types of classes of problems. Um, if I think back in my life, I have spent some months in making bespoke software development.

And, um, there usually is this this discussion should I make, or should I buy? And I think that this is also something that, um, comes into play here as well. Where are these, um, these clouds, these service clouds well chosen to solve a problem. And where is bespoke point solution like development of solutions?

Um, the better way to go. Is there a rule of thumb to look at that? When should somebody look at these AI service clouds? So that's a really great point that you bring up that these AI service clouds are meant to, um, serve a wide variety of types of needs, not even simply needs, but types of needs.

You know, if you're going to go into, um, into the AI vision direction, into language, um, into cognition, into, um, prescriptive or, um, uh, predictive analytics, um, AI service clouds can typically take you in any of those directions. Um, but on the other hand, there are point solutions that are bespoke, um, AI solutions, which are out there provided by vendors that, um, meet a specific need.

So, um, if you have a use case, which is very clearly defined, um, and the requirements are very stable, um, then looking at a point solution may be a good option. So, um, for example, in healthcare solutions, um, for detecting perhaps a malignant tumor in imaging, um, you know, these types of models and a situation in which it would be used are very tightly regulated and very, very specific, um, and also typically consistent from implementation to implementation.

So, uh, this type of use case, it's probably better suited to a point solution to, um, looking at a vendor who specify to specifically designed a model for this exact use case. Um, but if you're looking more at streamlining internal processes of a business for having a bespoke crossover between different departments, um, and for customizing a model with, um, your specific data with your specific environment, um, then an AI service cloud makes it a little more sense. It gives you a little more creative control, um, to, uh, to fit it to your environment.

Um, and so that's not to say that AI service clouds are going to replace a point solutions, um, or a specific, um, uh, solutions targeting a use case. Um, but they're going to supplement the market and they're gonna make customization much more feasible for those enterprises who have not previously invested in an AI development team. Right.

And, um, as we talk about this today, there is of course, a reason for that you did some extensive research around that topic around that rather than new market of service offerings and that results in a market compass document here at cooping, our code. So that will be soon out or is out. I have to ask.

Yes, it's out already. It was published, um, earlier last week, and yeah, there were some interesting findings from that, as you said, it is a new market.

And so, um, the vendors which are in it are, um, offering different approaches to, um, define what is an AI service cloud. And what does that, um, how is that going to be most useful to organizations?

Um, so a theme which came out is that communication with different stakeholders is really key. And so of course your stakeholders could be within your team, um, or outside into the world to those end users or perhaps people who are affected by a decision, um, which would be made by that model or with media, with, um, with external interest groups.

Um, so one interesting, um, development or feature, which is out there, um, are called model cards or some variation on that name and what it is is it's a, um, it's a summary of all the important information about a model, which is in use. So it includes some of the boring details, like what model it is exactly.

Um, and which data sources were used for it. But, um, especially for implementation, it also carries with it best practices.

So again, you imagine a model which is being used in a hospital by doctors on an emergency room floor. Um, they're going to be able to have access to that model and say, okay, this is best fused.

Um, for, um, you know, this has best accuracy for patients under 18. Um, and so that's a piece of information wouldn't they, that they should keep in mind as they look at the decision, um, provided by it.

Um, so that is a really useful, um, communication piece, which we'll probably continue to develop and improve over time. Um, another interesting, um, finding from the market compass report is that the modular report, um, the modular approach.

So being able to dive into one specific, um, point in the model development life cycle, um, means that you can hone in on explainability during, um, during the training and validation phases, or take the bias mitigation and drill down on the data preparation stage, um, where they can jump to a phase in this development, um, lifecycle where they don't have direct experience. Um, so that means if you have a team of data scientists, they're going to be great at that data prep stuff. They can do that all by hand, um, not by hand, but, um, with their expertise.

And then, um, use the semi-automated parts of the AI service cloud for later stages like training and validation, um, things like that. I think of what you described before, when it comes to, um, bringing together diverse teams, teams with people of different qualification, with different expertise. I think there's still a way to go for getting those people who are not the ML experts to get them up to speed, to give them training, to give them guidance, to give them best practices.

So I think it's not only just a technology as a piece of software, there is still some trainings, some, some support, some, some second level people required that guide organizations in their usage of this AI service cloud. Yes, that's absolutely true. So we've got a long way to go.

Um, but the, uh, AI service clouds are doing what they can and, and they've started this process to, to bring together diverse camps. Um, what some of them have as well is, um, are different interfaces for different, um, uh, participants in a project. And so if you happen to be, um, a machine learning developer, an expert, you can be working in Jupiter notebooks, you can be having all the control, which you would normally have.

Um, but then when you want the input from somebody else on your team who doesn't have that experience, um, they have access to the same project, but through a different interface, which is more drag and drop, um, which, uh, sometimes has the support of wizard to guide you through what you should do step by step. Um, so that's an interesting feature which does help to kind of build out the accessibility, right?

So you're looking at some kind of consumerization of machine learning, moving it away from this more academic, this highly technological point of view towards a more market relevant approach here, right? Yeah. And we'll see where it goes, you know, this, uh, this areas going through very rapid development.

Um, and so if we come back to this topic at this time, next year, it's probably going to be look, um, be looking quite different As this is a fast moving market. Um, it's great to have this document out already and to have this market covered with a first overview of this, of this solutions.

Um, thanks again, any for telling me and for telling our audience more about this exciting development, bringing machine learning closer to the organizations, any final recommendation, something that you would like to share with the audience, um, when it comes to looking at this AI service cloud model, is there a starting point apart, of course, from our market conference, which is on our website, um, I'd be interested.

Um, you know, there's a lot, um, coming out in the news at the moment, um, different, interesting projects, which, um, are being embarked on both in the private and the public sector. So stay interested, feel inspired, um, and know that AI is becoming more and more accessible. Exactly. It's close. It's getting closer to us. It's getting closer to the organization in the end. It's getting closer, um, to, to the, to the end user. And it's more, more usable. Thanks again, any for spending your time with me today for telling me about that and looking forward to seeing you again soon.

Bye bye Annie. Bye Matthias. Thank you.