This view of flow management...

On the origin and the descent of managing the flow of work. We put meat on the bones.

AI for Service Request Management

by Leave a Comment

In this second article in my series on artificial intelligence and service management, I will pro­vide a concrete example, name­ly, using AI to help manage service requests and other support re­quests from service con­sumers. I will apply the framework de­scribed in the first article in this series, A Model of Artificial Intelligence in a Service System. The example I provide in this current article will provide il­lu­stra­tions for many of the fu­ture articles in this series.

artificial intelligence phase model
Fig. 1: The life of an artificial intelligence in a service system

AI Applied to Service Management

Let me first set the stage by speaking more broadly of service management. Artificial in­tel­li­gence can support many aspects of designing, implementing, o­pe­rat­ing and improving services. Here are some examples of ap­pli­ca­tions which might usefully include AI components:
  • Identifying opportunities for services within market spaces
  • Optimizing the structure of complex service sys­tems
  • Optimizing the operations of service systems, such as minimizing energy con­sump­tion in a data center
  • Automated testing of service systems
  • Automated changing of components of service systems
  • Automated classification of service requests
  • Identifying operational risks
  • Identifying faults
  • Identifying potential cau­sa­li­ty
  • Segmenting customers
  • Analyzing emotions of ser­vice stakeholders, es­pec­i­al­ly con­su­mers and users requesting support
  • Tuning of services to improve customer sa­tis­fac­tion
A brief word about areas where AI is probably not (yet) very useful:
  • designing new and com­pe­ti­tive services
Whereas an AI might conceivably piece together the elements of a service system—even this is probably much too complicated for existing AIs—the result would probably resemble existing ser­vices and have no competitive ad­van­tage.

An Example from Service Request Management

I have chosen the classification of the various sorts of requests and other communications made by service consumers to a service providers to exemplify the life of an AI in a service system. In this article, I will talk of service requests and service support in­ter­changeably. I have chosen this example because it is within the experience of most service ma­na­gers. Furthermore, artificial intelligence has already been implemented in certain tools to help achieve this purpose.

service request classification
Fig. 2: Positioning service request classification (not intended to be a complete process)

The particular purpose of the AI will be discussed in more detail below. I am assuming that an AI is used because the service provider had been having sig­ni­fi­cant difficulties in resolving va­ri­ous types of support of its service consumer. Often, one of the factors leading to long lead times in resolving support re­quests is a high error rate in classification of the types of support being requested.

Let us examine how AI might be used to improve how a service provider might manage service requests.

The Cultural and Strategic Context of the Organization

In assessing the impact of the service provider’s culture on the use of AI for service request management, I will consider org­a­ni­zational maturity, es­peci­al­ly as it concerns its capability to reorganize itself and the levels of risk associated with the problem of misclassification of service re­quests.

Organizational Maturity

A relatively immature org­ani­za­tion might address the problem of service request misclassification by layering technology on top of its existing system, rather than seeking to resolve the problem at the level of its underlying causes. This is a classic result of a command and control culture. A management problem is detected; controls are added to address that problem.

For example, mis­clas­si­fi­ca­tion is a problem if it leads to assignment of tasks to the wrong teams. But this is a problem especially if the teams are organized according to the technology they manage, rather than according to the end to end services they provide. However, even if the erroneous assignment and the inefficient hand-off problem with technology silos were recognized, you might consider it extremely difficult to change that organizational principle. Many organizations are simply not ready for or lack the agility needed to make this sort of major change.

The maturity of the service provider organization impacts its understanding of how to manage the AI through its life. For example, if an organization naïvely believes that an AI can classify service requests out of the box, with no training, it will quickly be disabused of this notion. Understanding why a first line of support has difficulty in classifying support requests requires deep knowledge of the provider’s service system. This knowledge cannot be provided out of the box by a third party. At best, a tool can provide an algorithm for classification, but this al­go­rithm needs to be trained by the service provider.

Risk

What is the risk to service consumers of a mis-classified request? In a rigidly structured,  or tightly coupled system, the risk is that the wrong service will be provided or that the provider will attempt to resolve the wrong thing. The consumer will be greatly dissatisfied and the pro­vider will have wasted con­si­der­able effort and resources. In a more buffered system, mis­clas­si­fi­ca­tion is likely to be detected before the service is provided to the consumer. Thus, the risk is principally in extending the lead time for delivering the correct service. This risk exists for the first case, too, where it would be greatly aggravated.

As we will see, an AI delivers probabilistic results. In other words, each time it classifies an service request, it does so with an associated probability that the classification is correct. For a properly managed AI, such errors are easily detected and changes made to reduce their probability. Consequently, the risk to the consumer of using an AI for classification is low. The risk to the provider is in relation to the expense of creating the AI, which is directly related to its maturity.
Misclassification time-line
Fig. 3: Risk of mis-classification's impact on resolution lead time

Strategies Impacting Service Requests

The service provider will also need to establish policies, be they de jure or de facto, regarding the supplier of the AI technology and the technology itself. For the case of service request classification, this issue is essentially the old “best in class” versus “integrated solution” discussion. A stand-alone clas­si­fi­ca­tion tool could be integrated with the other tools used to support service request management. The question is whether the potential benefits of a stand-alone solution (namely, improved classification) would outweigh the costs of developing and maintaining the integration. This decision is further influenced by the capabilities of the service provider to perform that integration.

In the section below on Training I will address a particular issue that may impact the strategies for sourcing natural language pro­cess­ing. If you use multiple NLP back-end services, you will need to train each of them separately. You might find this necessary if you use an integrated service management tool with some NLP functionality, but you also want to use NLP for other purposes not covered by that tool.

Data Strategies and Service Requests

In a future article I will discuss the general issues of data stra­te­gies as they related to service delivery and service ma­nage­ment. Here, I will limit myself to remarks that directly concern service requests and service consumer support.

Pre-AI data strategies for service requests are generally based on two use cases. First, it is assumed that many support requests are highly standard in nature. For these requests, models are defined to collect structured data and to limit the amount of data collected to what is deemed essential for quick and efficient resolution.

For example, to resolve a request to restore a backup of a file, the model would tell the provider to collect the following data:

  • What file?
  • When was the backup made that should be restored?
  • Who is authorized to request a restore?
  • Who has made the request
  • When the restore should be made (if not as soon as possible)?

Any other data associated with the request is deemed to be super­flu­ous, ple­o­nas­tic or super­ero­gatory, hence dilatory and in­ef­ficient.

At the other extreme are the cases of unexpected and hitherto unheard of requests. Assuming such requests are within the scope of the support provided by the service provider—which is not always the case—emphasis is placed on collecting unstructured, descriptive or nar­ra­tive data. Some org­a­ni­za­tions will attempt to provide a partial structure for such data, such as describing a request in “user story” format:

The requester, who is making the request in her or his role as a [the role being played],
is requesting [some change, action or correction]
in order to achieve result or benefit [why the request has been made].

In both of these cases, there is an attempt to reduce the amount of data collected, assuming that structured, or semi-structured, data is somehow better than unstructured data. Whatever the pros and cons of structuring might be, this strategy of sim­plification and structuring also re­sults in a loss of data. Assumptions are made about what is important. There is a tendency to disregard and not record other data. For example, information about the emotional state of the requester is often not recorded. The support agent responds to emotions either in trying to sedate them; empathize with them; ignore them (because they assume that only the “rational” component is worthy of attention); or they might find it expedient to raise the priority of a request just to resolve more quickly and dispense with the unpleasantness of the case.

When using AI to support work, err on the side of collecting more data rather than less data.I am not arguing that structured data should not be collected. Nor am I arguing that unstructured data should not be structured, to the extent possible. However, I am arguing that these models and structures should not auto­ma­tical­ly exclude certain types of data associated with support requests. Indeed, I would argue that a strategy should be implemented to attempt to collect more, rather than less, data, making it available to AIs that would support the analysis and fulfillment of  requests.

For example, when a request is registered by telephone, the entire conversation might be recorded and made part of the request record. Such recordings are useful for more than training purposes or for evaluations of support agents. They can help reduce the impact of the biases of support agents regarding what is important. They can help make objective the analysis of requester intent and emotional state. Such information might be useful, for example, in the assigning work to agents. Certain agents might be more skilled than others at handling certain emotional states.

Defining the Purpose of the AI

First of all, the scope of the AI must be clearly stated and constrained. Suppose you wish to improve how requests from service consumers are handled. In the current state of the art, you cannot design an AI whose purpose is to “manage service consumer requests.” You can, how­ever, decompose that ma­nage­ment into detailed tasks. Some of those tasks might be good candidates for automation using AI.

It is often the case that tasks are not correctly assigned to teams. This can result in huge amounts of waste and dissatisfied consumers and providers. Thus, one example of such a task is the classification of service requests. What is the problem that organizations hope to solve? Often, the service provider is organized into teams of specialists, based on the principle of technology. Thus, there are network experts, server experts, client experts, database experts, application experts, etc. I set aside the fact that such an organization might be extremely inefficient for handling the flow of work.

Before described to steps to build and deploy an AI for service request classification, I will briefly review some of the other strategies that have been used to solve the problem of poor classification.

Other Strategies for Classification

In deciding whether an AI best meets a defined purpose, it is useful to review other possible solutions. Several strategies have been tried to address the problem of classification:

  • Offloading classification to the service consumer
  • Improving the classification skills of the first line of support
  • Implementing tools to enhance communication be­tween teams
  • Using simplistic classi­fi­cation schemes, often based on components involved

These common strategies have all various difficulties:

Offloading Classification

Offloading clas­si­fi­cation to service consumer, otherwise known as self-service, often requires complex taxonomies of request and support types. The user interface often navigates the taxonomy via a series of hier­archical drop-down lists. The complexity of this interface and taxonomy befuddles the non-specialist consumer, who has little motivation and no time to learn the details. Thus, captured clas­si­fi­ca­tion is often too general to be useful and often wrong. The poor quality slows down handling because it requires a secondary con­trol and qualification to correct errors and provide needed detail.

Improving Classification Skills

Improving the classification skills of the first line of support sounds like a good approach. It embeds knowledge within the organization and helps avoid the waste of misdirected tasks. Un­for­tunately, this strategy is often sacrificed on the altar of saving costs. First line of support personnel are often outsourced, resulting in higher turnover rates and a failure to embed knowledge within the service provider or­ga­ni­za­tion. They are treated as low-level jobs that do not merit good salaries, thereby adding to turn­over rates and discouraging people with the interest and the skills needed to do a good job of classification. In addition, the transfer of the knowledge required for good classification from the technical teams to the first line of support is often considered to be a “nice to have” task in any project. Consequently, it is among the first tasks skipped when a project is inevitably taking too long or costing too much.

Enhancing Communications Tools

It has become fashionable in the past few years to implement tools to facilitate inter-team com­mu­ni­ca­tion. The benefits of such a strategy are based on the astonishing proposition that peo­ple who have communicated poorly outside of their own teams would start to communicate well, if only they had the right tools. It might be true that the novelty of implementing the latest technology will attract the attention of died-in-the-wool technologists. It might also be true that young employees, who have been using a certain com­mu­ni­ca­tion technology since their tenderest of ages, will happily continue to use such technologies in the workplace. But adding a layer of technology to foster communication increases the degree to which work is interrupted. Interrupting work and the associated context switching in the brain are poor ways of reducing lead times for work. Instead, they increase the waste of context switching and the stress of trying (but failing) to work on multiple tasks at the same time. Thus, while com­mu­ni­ca­tions tools might show some benefits, they also bring with them a new set of problems to solve.

Using Component-based Classification Schemes

Using component names or classes for directing tasks has a long history. Essentially, this approach assumes that the service provider can know the principal component implicated in a sup­port request and that component is supported by a particular team. Therefore, the team to which the request should be assigned is determined looking up the com­po­nent in some configuration database and re­trieving the name of the responsible team. In addition to the issues described above (especially the issue of insufficient training of the first line of support), there are three problems with this approach:
  • the actors in the initial exchange of information about the request are often not aware of more than the consumer’s device, the application being used and the edge networking connections
  • issues often need to be resolved elsewhere in the technology stack
  • errors or incomplete information in the configuration database may lead to misclassification

Building the AI

So the service provider has decided to improve classification, rather than attempting to remove the underlying issues that make misclassification a problem. To build an AI to help improve classification, it needs to gather relevant data, model how that data will be processed and perform the initial training of the resulting AI. These steps are highly iterative. Furthermore, they include testing, as ap­pro­pri­ate to the culture and maturity of the service provider organization.

Gathering the Data

What data is available for the classification of service and other support requests? In general, a ser­vice provider will have:

  • the record of the initial request by the service con­sumer, including its initial classification
  • a catalogue of types of service or support requests
  • a database of service system components, possibly show­ing their interconnections
  • an organizational structure describing all the entities to which a request might be assigned
  • records of the patterns of assignment and reassignment of tasks to handle the requests
  • possibly some form of feedback from the actors involved about what worked well for classification and what did not

Since the purpose is explicitly one of classification, it will be necessary to use labeled data for the training that will follow.1 In other words, for each record of a service request used for modeling and training, the correct classification (and how does that classification results in assignment of the request to the correct team) must be recorded in advance of the modeling and training exercises.

Now, we would never think of developing an AI for the purpose of classification unless there were a considerable degree of mis­clas­si­fi­ca­tion. Therefore, it is important that the data sample used for modeling and training be reviewed and corrected by subject matter experts.

You might be tempted to use shortcuts to address the problem of bad data. One shortcut is to arbitrarily filter out data from sources that appear to be unreliable. This approach is likely to introduce significant biases to the modeling and training data. Another shortcut is to handle the problem of bad data in the live environment, as part of the continual tuning and improvement of the AI. While this approach might work, there is also a strong risk of the AI’s performance being too poor at the start to convince the stakeholders of its value. As a result, the whole effort might be abandoned before it starts to show much value.

Failure to do a review of data quality will result in an AI that simply repeats many of the same mistakes that existed in the past. This is particularly im­por­tant because the dis­tribution of request types is usually highly skewed. Thus, nearly 1/3 of all requests might be for password resets. The top ten requests might account for as much as 90% of all requests. But these requests are the easiest to classify and the least likely to be in error.

So, a very large number of requests occur relatively rarely. Inversely, most of the errors in classification are likely to occur among the rarer types of requests. To use an analogy, a physician is probably a pretty good diagnostician for illnesses that she or he sees every week. Diseases that might never have been seen before, or only very rarely, are much harder to diagnose accurately, even though they might tend to be more debilitating.

Typically, the data should be split between training data and testing data. Since service and support requests tend to be highly cyclic and seasonal, both sets should include many examples from all parts of all cycles and from all seasons. If this data is not available, Then it might be possible to use live data to train or to test. If there are annual cycles in the data, then training on live data might take much too long. The only alternative is to use synthetic data, generated in a random way, much as a Chaos Monkey can generate random faults in a live system.

It is probably not a good idea to use historical data that is too old. This is because there are trends in the types of requests being made, changes in organizational stru­cture, drifting in the jargon used by the organization and changes to the services themselves. Training on data that is too old might result in irrelevant classifications.

This is a good time to define metrics for the quality of the classifications and to make baseline measurements, before the AI classification component is implemented. You should also set a target for improvement in these metrics. Some of the metrics that might be useful are:

  • How long does it take to classify correctly
  • What percentage of requests are classified correctly
  • How much does it cost to develop, operate and maintain the AI

Modeling the AI

Modeling the AI should take into account all the structured and unstructured information avail­able. This would include the time and date of the request, the full text of the request and any descriptive material available, the identity of the person on whose behalf the request is made, the service system components a­vail­able to that person (services, devices, etc.). It might also be the case that the e­li­gi­bility for services is contractually defined. In this case, it is important to have a description of that e­li­gi­bi­lity.

In all likelihood there will be a gap between the data available for modeling and the data that the modeler would like to have. Recognize, then, that the initial model is likely to evolve as more data become available in the future. This is especially true if the virtuous cycle2 of AI use en­cour­ages improvements in data strategies.

Every service provider will offer a different set of services, have a different set of service consumers and different types and amounts of data concerning the service requests. Applying a generic model developed elsewhere is likely to yield highly mitigated results, except for the obvious cases where no one really needs an AI to help support the work.

The work of developing a model is highly technical. I will not go into the details of how to do it. Fun­da­men­tally, it involves identifying the factors that strongly indicate the desired classification. A common modeling method will determine some form of regression of the data. The end result of the model will be a means of assessing each of the components available as input for the classification of the request, providing relative weightings to the different available inputs, and describing how to combine them to determine the final output, the classification.

Let’s examine a few examples of requests and what issues need to be handled by the model.

I was trying to log in the FI module of SAP but it rejected my cre­den­tials. It’s been a few months since I logged in and I seem to have for­gotten my password.

So what are we looking at there? Is it a simple case of faulty user memory and forgetting a password? Does the person even have an account in that application? Is it possible that someone has pirated the account and changed the password? A natural language processing unit will parse the text and identify certain key entities, such as “SAP”, “password” and “forgotten”. The AI might also compare the identity of the caller to the authorizations, according to LDAP records or even SAP records.

I was performing the annual closure function in Whizzy when the following error message ap­peared…

Many tasks, and their related data, are highly sea­sonal. The timing of the request within the seasonal cycle might have a strong impact on its clas­sif­ica­tion. In addition, many of the components of a service system are idiomatic. “Whizzy” is the org­an­i­za­tional jargon for a certain application. Thus, the model cannot depend ex­clu­sively on a standardized list of component names.

While working on my com­puter, PCGV0014, eve­­ry­­­thing was fine until a­round 11:00, when the screen froze and no longer re­spon­ded to the key­board. I had been using Calc in Libre­­Office at the time.

The narrative provided by the service consumer mentions several entities, including the computer in use and the application being used. The model might take into account the possibility that some of these entities are irrelevant to the issue and that there might be a hierarchy of components. Misinterpreting the in­for­ma­tion provided might result, for example, in assigning the task to a desktop technician, where­as the real issue is a bug in an application. On the other hand, knowing the platform and being able to look up its configuration might result in sending a technician to upgrade the device.

After logging in to the account xyz, the file that was opened did not show the up­­dates per­formed ear­l­i­er this morn­­ing.

The model should be alert to the possibility that the request concerns a third party. Some systems include this in­for­ma­tion among the structured data of a request, but that data is not always reliable. Other systems rely wholly on the un­structured request narrative to provide this fact.

Training the AI

The model consists of a series of so-called “activation functions”, among other components, each function running in each node of the artificial neural network of the AI. These functions progressively transform the inputs (the struc­tured and unstructured data used for training and testing) into an output (the probable class­i­fi­cation). The training phase tunes the parameters of these functions so that the AI becomes increasingly reliable in de­ter­min­ing the desired output. Ideally, as training progresses the output values converge on the desired results.

The weighting of the parameters reflect the artificial neural net­work’s probabilities of the possible intermediate and final outputs. Let’s take the example above of a computer that crashes while using a certain application. By correctly classifying the event, it will be assigned either to a technician who will work on the computer, on site, or to an application administrator who will in­ves­ti­gate possible con­fi­gu­ra­tion errors or bugs in the application. A wrong clas­si­fi­ca­tion will sig­ni­fi­cantly increase the lead time to resolution.

Levels of Confidence

The purpose of training is to determine the most probable classification, given all the inputs provided. Thus, each output is associated with a confidence level, ranging from 0 (impossible) to 1 (absolutely sure). The AI will output what it esteems is the value in which it has the highest level of confidence. In practice, confidence levels above around .7 are considered to be very highly probable.

The level of confidence differs from human classification only in terms of its transparency. Unless an output is obvious and trivial or can be calculated by some un­am­biguous formula (in which case there is no need for an AI), human classifications also have a probability. When we say, “I am sure the classification is x”, what we really mean is that my confidence in the classification is high enough that there is no point is doubting it. When we say, “I think the classification is y”, we are saying that we are confident enough to express an opinion, but we are not sure. Often, we refuse to express any opinion at all because we are not sure of ourselves. When we factor into these classifications the various cognitive biases of the person and unstated assumptions, we see that the user of the classification should only have a certain level of confidence in the human-generated classification. The difference with the AI generated classification is that the level of confidence is explicit, whereas with human clas­si­fi­ca­tions, we rely on unstated assumptions to assess our con­fidence. We implicitly think “That guy is very reliable. I trust his judgement” or “The last time I depended on his answer, I was led astray. I better double-check this time” and so forth. And thus we add our own biases on top of the classifier’s biases.

After training, the input of this example might result in an assignment to the desktop technician with a confidence of .2, an assignment to the application administrator with a confidence of .75 and the remaining .05 distributed among other as­sign­ments. Consequently, the AI will output a classification that leads to assignment to the application administrator.

On the other hand, the way in which the training leads to this result is not very transparent. It takes into account all the inputs and the defined model so as to make its pro­ba­bi­lis­tic cal­cu­la­tions. The training is not like the rule-based analysis that humans might use, analyses subject to very many biases. That does not mean the AI is unbiased. It is subject to all the biases in the selection of the training data, in the model itself and in  the biases of the subject matter experts who labeled the training data with the desired classifications.

Training a Natural Language Processing Component

Suppose you wish to classify a support request that has been registered by a service consumer using email or a chatbot or even a telephone conversation that has been recorded. A natural lan­guage processing component is needed to parse the text that has been recorded. This component requires training. While an NLP component is likely to have a basic understanding of grammar and syntax of the language being interpreted, it cannot know out of the box the jargon in use in your organization. This jargon describes various service system components, your services, your organizational struc­ture, the multitude of acronyms that are likely to be in use, and so forth. A given NLP service might even recognize certain entities commonly used in service systems. For examine, it might recognize that “SAP” is an application suite, in addition to being the fluid flowing in a tree’s xylem. To train the NLP component you should define the key entities that need to be recognized together with the synonyms of their canonical names. For example, each software application in use has a canonical name given by its editor. It probably also has the name to which its users refer, which might be different from the name used by its administrators. If the name is long, it might be abbreviated with an acronym. And very an application’s moniker is simply the name of the company that created it. All of these different names must be recognized by the NLP as referring to the same entity.

Integrating the AI

Since our AI has a very circumscribed purpose—to clas­si­fy service requests—it serves no purpose in isolation. It must be integrated into a broader service management system.

While you might favor a solution where the AI function is part and parcel of a general service management tool, this is not always an option. When im­ple­mented as an external tool, that tool should be in­ter­faced with the various sources of data described above. It might also be desirable to realize the AI as an independent tool if the general service ma­nage­ment tool lacks the desired functionality or ef­fec­tiveness.

For more back­ground in­for­ma­tion about these integration issues, refer to the discussion above about strategies.

Deploying the AI

There is a distinct possibility that there is insufficient training data for effective classification. In this case, the AI might be rolled out in phases. In one approach, the AI uses the live data as an ongoing source of training data. The automatic setting of classification is not enabled until sufficient training is obtained.

In the case where the first line of support is not centralized, there might be a roll-out team by team. The first team to use the AI would also serve as a pilot demonstration of the technology, allowing for additional ad­just­ments to the mo­del and the training before fur­ther deployment.

Operating the AI

The operation of the AI may take one of several forms, depending on whether the service consumer is making requests to a self-service portal or to a human support agent. In the case of a self-service portal, the most likely interface will be a chatbot, whether typed or oral. The unstructured data to input to the AI will be the entire transcript of the chatbot session. Note, though, that some chatbot tools can be configured to gather structured data, too. In the case of making requests directly to a human agent there are several possibilities for capturing the narrative of the request. One possibility is the traditional method wherein the agent types a resumé of the request into the support tool. Another possibility is that yet a different AI auto­ma­tically transcribes the conversation, if it is oral. Finally, communications via chat tools, email, etc., would capture the narrative in much the same way as for chatbots.

Once the data of the request is captured, it is fed to the classification AI. In the case of registering a request via the mediation of tools (chatbot, chat, email, self-service portal, etc.), this would be done automatically. In case the human agent registers the request manually, the triggering of the classification could be manual or automatic, depending on the type of integration and the interfaces in place.

Finally, the AI arrives at a clas­si­fi­ca­tion and its associated con­fi­dence level. Given the pro­ba­bi­lis­tic nature of artificial neural net­works, it might be a good idea to have a final confirmation step before recording the clas­si­fi­ca­tion. This would be possible in any form of interactive contact with the service consumer, such as via a chatbot or a telephone con­ver­sation. The chatbot or the human might say, “This is how we have understood your request” and request some confirmation by the consumer that the clas­si­fi­ca­tion is accurate.

I noted above that an AI will output a classification with a certain level of confidence. This means that each service provider organization must judge what level of confidence is sufficient for its purposes. This judgement depends on the organizational tolerance of risk, the trust it has in the technology and the skill with which the AI has been modeled and trained. This view of levels of confidence influences the par­ti­cular way in which the AI is operated.

The final step is the recording of the classification. Depending on the level of confidence and the threshold established by the organization, this may be done manually or automatically. Typically, an automatic record would be made for high levels of confidence. For low levels of confidence, the classification would have to be referred to a human agent. There is probably a grey zone between the two, wherein the AI might suggest a classification, but a human agent might adjust it. In any case, human adjustments to the classification should always be possible, but it would be useful to record the fact that a human has overridden the AI. These data will be used during the continual learning phase.

Improving the AI

There are many reasons why an AI needs to continually evolve and improve, via some feedback and learning. Among the obvious reasons are changes to the services being offered by the service provider and changes to the types of support available, such as the support channels and the levels of service. More subtle are the changes to the consumers, whose skills and expectations may change over time; changes to the jargon in use in the organization; changes to the competitive position of the service provider and the need to improve its services.

It is also likely that the service provider will come to recognize the usefulness of data types that had not been collected at first. As a result, effort is invested in enlarging the data available for classification purposes. Fur­ther­more, there might be im­prove­ments in the quality of the data available, such as more complete and more accurate con­fi­gu­ra­tion ma­nage­ment data.

There are various types of data that should be logged, which data will be the basis of the im­prove­ment ac­ti­vi­ties.  I suggested above that clas­si­fi­ca­tion should be confirmed interactively with the consumer, if possible. Cases where the consumer responds, “No, that’s not it at all” need to be recorded. This is prima facie evidence that the AI is not classifying correctly for the inputs it has received.

In some cases, records will be reviewed and adjusted after the service request is resolved. If the classification needs to be changed, given the full experience of handling the request, then this, too, should be fed back to the AI to make improvements.

Another type of data used to improve the AI is the record of rejection of assignments and reassignments of tasks. When a request has been misclassified and, as a result, assigned to the wrong team, this can be detected from the assignment records (assuming a push-type method is in use for managing the flow of work).3 This sort of issue is distinct from the problem of playing ping-pong with an assignment, which may indicate another type of problem, not necessarily a classification issue.

Feeling the effect of feeding back additional data depends on some parameters in the configuration of the AI, such as the “learning rate“. Essentially, this parameter adjusts the sensitivity of an artificial neural network to changes in training data. This, in turn, affects the time needed to train, the amount of training data required and even the ef­fec­tive­ness of training in converging on stable and reliable outputs.

Let me give an exag­gerated example. An important question is how to weight the mention of a desktop computer name relative to the request classification. For some requests, such as adding a new screen or replacing a broken mouse, it might be extremely important. For other requests, such as restoring the backup of a file to a network drive, it is largely irrelevant.

Now, imagine the first training data set mentions a desktop computer name and the case is labeled as “Repair desktop computer”. The artificial neural network will decide that the computer name is really important. But suppose the next data set in the training data also mentions the computer name, but is labeled as “Restore file from backup”. If the learning rate is set too high, the the AI will think, “Oh, that last example was probably just noise. Instead, the desktop computer name seems to be a good indicator of the need to restore a file.” And so it can flip-flop back and forth because it is too sensitive to the changes in training data.

But suppose the learning rate is too low. If the first training example is labeled as “restore a file”, it might take a very large number of counter-examples before the AI finally converges on the computer name being a good indicator of a repair issue and a bad indicator of a file restore issue. So, the art of setting the learning rate, which is a technical task under the re­spon­si­bility of the data scientist, can influence the cost of the AI, its time to market and its effectiveness in improving the classification of service requests.

Communicating with Stakeholders

In the previous sections of this article I have portrayed the example of AI applied to service re­quest classification. Certain communications with service request stakeholders should help to frame the expectations of ap­ply­ing AI to this purpose, jus­ti­fy its use and support the success of an AI approach.

Fig. 4: Overview of communications types regarding the use of an AI. For the sake of simplicity, I do not show consumer-provider service co-creation communications.

Communicating with Service Consumers

Some service providers might believe that using AI is an end in and of itself. They believe that it is “modern”, “up to date”, “com­pe­ti­tive” or perhaps even “mi­ra­cu­lous”. While this sort of hype might be eye-catching and draw some at­ten­tion, I am more concerned with the longer-term value-related issues that underpin a successful provider-consumer relationship.

Justifying the Collection of Data

The success of using AI in the management of service requests is likely to depend on the collection of data about the service consumers. If those consumers are external to the service provider, many countries require a great deal of transparency regarding what data is collected, how it is used, the rights of the consumer regarding those data and the responsibilities for handling the data. Thus, the service provider might be legally required to make all this information available to the consumer before the consumer engages in any transactions.

I believe that service providers also have a similar moral ob­li­ga­tion toward its employees and con­trac­tors, even if no laws oblige such transparency.

Communicating Changes in Interfaces

I will treat the different communications channels se­pa­rate­ly: telephony; on-line self-service; and asynchronous off-line com­mu­ni­cations. The service con­sumer should probably not expect any changes in the nature of making a request via telephony. However, to the extent that the conversation might be subject to speech-to-text handling, the con­sumer should be made aware that the conversation is being re­corded, as a precursor to the classification treatment.

The self-service interface might have a more significant interface change. Traditionally, self-service interfaces attempt to structure what is fundamentally an un­struc­tured communication. Thus, con­su­mers are sometimes led through a series of drop-down menus or other mechanisms to guide them through a complex taxonomy of choices. Such interfaces often have mixed results, largely because they are unnatural ways of communicating and they tend to view the information taxonomy from the service provider’s perspective rather than from the service consumer’s perspective. In the end, these interfaces only collect what the service provider wants to hear, rather than what the service consumer wants to say. As a result, much potential information about the request might be lost.

A more natural way of communicating is the free speech approach of the chatbot.4 By collecting data that the consumer wishes to communicate, the channel is less apt to filter out useful information. Too, it allows for disambiguation of speech, helping to identify key entities in the communication.

In some contexts, service consumers have been passively trained to communicate in a certain way to service providers. This bias may lead them to present what they believe the provider needs to hear, rather than what the consumer wants to say. Therefore, it may be im­por­tant to provide signals to the consumer to be at ease, to not make assumptions about “right” and “wrong” in­for­mation to communicate. Con­su­mers should not try to manipulate or trick the system to work in the their favor.

Communicating with Service Provider Personnel

The introduction of artificial intelligence tools is often perceived by employees as a threat to the nature of their work, to their identities and, indeed, to their very employment. Managers often encourage such negative reactions by portraying technology in­tro­duc­tion prin­ci­pal­ly as a way of reducing FTEs.

As we have seen, AI cannot be used as an end-to-end tool for providing or managing services. Today, AI can only respond to limited and well defined purposes. The major benefits of AI in service request management are in reducing lead times, reducing errors, increasing the satisfaction of all stakeholders and possibly reducing costs.

As a result, any initiative to introduce AI into the ma­nage­ment of service requests should start by identifying what the support personnel believe they need to improve the performance of their roles. Framed in this way, AI becomes a tool the existing personnel use to do their jobs better and increase their sa­tis­fac­tion in their own work. Support personnel remain the masters of their work, rather than being subjected to “im­prove­ments” by people who lack the experience of working on the front line to support the service consumers.

Except for the simplest and most obvious of cases, classifying service and support requests can be hard to do and takes a lot of time to perform correctly in a high pressure atmosphere.  The purpose of AI is thus to help the support personnel to perform a task with which they have difficulty. Com­mu­ni­cating this fundamental message to the support personnel will help reduce the antagonism with which many people greet imposed change. Instead, introducing AI is viewed as evidence of concern for the employees’ well-being and in­vest­ing in their capabilities. As the saying goes, it is not that people dislike change; they dislike being changed.

Communicating with Management

If, as they say, “AI is the new electricity”, then it is critical for the perdurability of the service provider organization to gain artificial intelligence capabilities. But AI for the sake of AI is sterile; it cannot be an end unto itself. The business purposes of using any AI tools must always dominate any communications with the service provider management. At the end of the day, AI must provide economical solutions to real business problems.

Communicating with Regulatory Authorities

Certain businesses are heavily regulated, due to the close relationship between the outputs of those businesses and the health and safety of its customers and of the environment at large. In a world accustomed to thinking in deterministic terms, how easy is it for regulators to accept a management system based on statistical learning?

Take the example of the manufacturing of pharmaceutical products. Given the risks to patient health of medicines that do not conform to authorized spe­ci­fi­cations, the entire manu­fac­turing system, including any changes to that system, is subject to regulation (at least, in many countries). It is required that calibrated tools used according to a known and authorized process, be used to reliably produce drugs of an acceptable level of quality.

Now, imagine that we introduce into that system a technology that does not claim to work in a deterministic way, but in a pro­ba­bi­listic way. Will that technology be acceptable to the regulatory authorities? If they ask you if your system will guarantee the health and safety of the consumers, will they be satisfied if you respond, “Yeah, probably”?

Until such time as a technology becomes a generally accepted practice that does not require ad hoc authorization, the users of AI should be sensitive to this issue in communicating with regulatory authorities. I think the best approach to take is to demonstrate a lower defect rate when using AI. In fact, the real issue is the belief that other systems are not probabilistic. In fact, all in­for­mation systems and manu­fac­turing systems have a probabilistic aspect.

Conclusion

In this article I have attempted to instantiate my proposed model for using AI for service management. I have used a simple example that should be familiar to most service managers. Each phase in the life of an AI, as described in that model, has been discussed in some detail, illustrating the principles in­volved. It is now up to you to apply those same principles to other cases where AI might help you to better deliver and manage your services.

Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International LicenseThe article AI for Service Request Management by Robert S. Falkowitz, including all its contents, is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Bibliography

[a]   Perrow, Charles, Normal Accidents. Living with High-Risk Technologies, Princeton University Press, 1999.

[b]   Hubbard, Douglas, How to Measure Anything. Finding the Value of “Intangibles” in Business, 2nd ed. John Wiley & Sons, 2010.

Notes
1 In principle, an AI would be able to cluster data without first labeling the training data. This might even be a very good idea. However, for the purposes of this example, where the taxonomy of request classifications and the mapping of those classifications to teams already exist, I assume that a supervised training approach, using labeled data, is to be used.
2 I will describe the virtuous cycle in a separate article on data strategies, AI and service management.
3 I remind the reader that the example used in this article was chosen for its ease of understanding by most service management professionals. I am not necessarily recommending that an AI be developed for the purposes described. In fact, I believe that the problems of classifications are mostly an artifact of a combination of using a push method for flow management and an organizational structure based on technology. If, instead, flow were managed using a pull method such as kanban and teams were structured according to business services, most of the issues of classification would simply not occur. There would be no compelling reason to develop an AI to support classification. There would be plenty of other uses of AI, as mentioned near the beginning of this article.
4 I have seen some chatbots, however, that are essentially means for collecting structured data rather than means to encourage natural language. I think the designers of such “chatbots” have completely missed the point of this communication method.
Credits

Unless otherwise indicated here, the diagrams are the work of the author.

Summary
AI for service requesta management
Article Name
AI for service requesta management
Description
The Model of Artificial Intelligence in a Service System, described elsewhere on this site, is exemplified by the case of classifying service requests
Author
Publisher Name
Concentric Circle Consulting
Publisher Logo

Subscribe to our mailing list

Click here to be the first to learn of our events and publications

Reader Interactions

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.