Organizing for Kanban

When Kanban decides against lower lead times

Certain organizations applying Kanban principles may choose not to include IT operations specialists in cross-functional teams, on the assumption that it would be very hard to keep such specialists busy. These organizations prefer, instead, to group the operations specialists in their own teams, thereby increasing the risk that an operations specialist might not be available immediately when a work item is assigned to that team. In other words, a compromise is made in favor of higher occupation levels for these specialists, as opposed to higher levels of throughput and lower lead times.

This article discusses the paradigms for organizing IT personnel that underpin such decisions. The choice between a pure cross-functional team and the separation of developers and operations into separate teams is more nuanced than a simple choice between optimizing occupation levels as opposed to lead time.

The Three Paradigms

There are three major paradigms in use today for organizing an IT department—or any entity, for that matter.

The first paradigm, one that some might call “traditional”, organizes according to technical aspects or technology (see Fig. 1). (NB: The diagrams depict organizations as hierarchies. This is purely conventional and is not meant to exclude any of the other types of relations that might order the entities within an organization. The advantage of the hierarchic diagram is that it is readable at a glance. The same cannot be said for matrices, congregations, federations, markets or any of the other forms that organizations might have.)

The second paradigm, fostered in part by agile development, is based on so-called “cross-functional” teams that are organized according to business domain (Fig. 2). A variant of this paradigm is organization according to product or product line. This type of structure is not readily applicable in the case of products or services offered across multiple business domains, as is typically the case for services such as messaging, desktop computing, and so forth.

The cross-functionality of the teams that are the units of organizational structure is depicted in Fig. 3. The specific technologies for which the team members have skills might vary from business domain to business domain. Similarly, the number of persons with those skills is apt to vary with each team. The diagram in Fig. 3 shows a structure where each person is responsible for a single set of skills. However, as we will see later, some or all of the team members might have multiple skills. For example, a single person might combine database administration skills with software coding skills. Another might combine system administration with network administration, and so forth. Note that the leadership of technology functions is not inherent in the organizational structure and must be determined by means other than the organizational structure.

A third paradigm organizes workers according to the processes on which they work (Fig. 4). This third paradigm is sometimes difficult to distinguish from the other paradigms, insofar as certain processes and certain technologies are used only for certain business domains. Note that supporting activities, such as financial management, personnel management or procurement are considered to be based on the process, rather than the business domain, principle. If someone prefers to view them as business domains, I will not argue the point.

Many organizations use a hybrid model of these paradigms (Fig. 5). Hybridization generally occurs either as a result of historical choices or as a result of pragmatic decisions based on available personnel, skills and management capabilities. However, certain trends are visible in the common hybrids.

The closer a team is to managing some aspect of a business application, the more likely is it to be organized by business domain. Thus, application developers are frequently organized into teams responsible for creating and maintaining the in-house applications for a given business domain. Similarly, the administrators of those applications are often organized according to the domains they support. At the other extreme are the teams responsible for low level infrastructure, such as the data and telephony networks, the data security systems or the directory services.

A typical example of a hybrid would be an organization that makes a high level technology split between software life cycle management on the one hand, and computer, network and other infrastructure management, on the other. However, within the software division, the lower levels of organization are apt to be defined on the basis of business domains, rather than, for example, programming language.

The typical implementation of the business domain paradigm, when used at the top level in IT, is the creation of “cross-functional” teams. (I use quotes because this term appears to make the unjustified assumption that a function can only be determined on the basis of technology.) Each team takes full responsibility for all aspects of one or more products or product lines. In order to do so, it includes people with knowledge of all the technologies requires, be they software development, database management, computing platforms, etc.

More recently, a third paradigm has started to appear in earnest. It is based on management discipline or process. In fact, this principle has always existed and is particularly common for such disciplines as IT finance, IT personnel management, IT architecture management, IT security management or project and program management. These management disciplines are typically not part of the direct value stream that IT offers to its customers. The novelty we are seeing is organizational structure based on such disciplines as incident management, problem management or change management.

In fact, the process paradigm has been rather common in the area of software lifecycle management. Thus, some developers are charged with creating new applications; other teams might have responsibility for fixing bugs or simple maintenance and changes to applications. It is becoming more and more common, however, to see service management offices and infrastructure teams specialized in certain processes, as mentioned above.

The distinction between a technology-based organization and a product-based organization is sometimes a question of emphasis or service orientation. This is typical of entities that manage technologies and offer services that are used across the corporate landscape, often described as “infrastructure”. For example, one team might view itself as the manager of message transfer agents and protocols such as SMTP, or of specific technical products, such as Sendmail, Domino or Exchange. Another team might view itself as responsible for delivering an electronic messaging service, independently of the specific technologies in use.

Advantages and Drawbacks

Each of these paradigms has its advantages and drawbacks. We may analyze the differences in terms of:

• knowledge management
• efficiency
• career development
• standardization.

While this list is not complete, it allows me to highlight the key issues relative to the use of cross-functional teams.

Knowledge Management

Each paradigm excels in the possibility to create and share knowledge about the organizing principle of that paradigm. Thus, a technology team is well placed to share knowledge about that technology, but this principle does not help in the sharing of business knowledge. A business team fosters a holistic understanding of a business domain, but has no special effect on sharing any technical knowledge that it might discover. A process-based team might know all about how to optimize that process, without understanding the business uses to which that process contributes.

Business domain orientation might have a salutary effect on the management of highly business-dependent areas that are often weak in the technology-oriented team. For example, the availability, capacity and security of systems depend heavily on understanding business domain priorities and drivers, which information is much easier to gather when the whole team is oriented to the needs of a business domain. Technologists might be tempted to say, “that’s not my department!”

Knowledge management in a process-oriented structure is a special case. A service management office, for example, might generate knowledge about a particular process and might foster the sharing of that knowledge among the owners or managers of the related processes. However, the usefulness of such an organizing principle depends on the dubious hypothesis that everyone in an IT organization should be using the same processes. I will discuss this issue below under the rubric Standardization.

Efficiency

Remember that efficiency is a measurement of the ratio of the resources and people used to the output of those resources and people. There is an extensive literature about the relationship between organizational structure and efficiency. However, the underlying assumption of much of this literature is that organizational structure may be altered for the purpose of controlling overall efficiency. The assumption is in conflict with the assumptions of a Kanban approach, wherein the reason for changing structure—ifreason there be—is to optimize the flow of work, not to facilitate the command and control of the personnel.

The great difficulty with efforts to improve efficiency is the assumption that changing the efficiency of a single person or team necessarily changes the overall efficiency of the organization. While this assumption sounds intuitively obvious, it is demonstrably false in many cases. On the one hand, changes in efficiency impact the overall process only if they occur at the bottleneck in that process. On the other hand, making sure that everyone keeps busy, creating as much output as possible, is a sure formula for creating a great amount of waste and ultimately raising costs while lowering overall lead time and throughput.

The main point of distinction between the business domain principle and the technology principle turns around the question of efficiency. If you believe that it is more important to keep everyone busy than to have the right resources available at the right time, then you are condemned to using the technology principle.  A simple example illustrates why.

Suppose you create a business domain team for financial management, consisting of a few developers, a DBA, a system administrator, some application administrators and various other persons. The system administrator will play a key role in designing and deploying platforms required by the finance systems. But outside of those periods of intense activity, that person may have very little to do.

The situation becomes more extreme when we consider that a single person cannot possible guarantee availability during all working hours. Absences due to illness, vacations, training and so forth prevent full availability. And so, perhaps it would be necessary to have at least two system administrators on each functional team—two persons who risk being idle much, or even most, of the time.

When many of the personnel are frequently idle, this is an intolerable situation for many managers. It is obvious to such managers that all the system administrators must be grouped together in a technology team that delivers services to all the other teams requiring centralized computing platforms.

Career Development

Many organizations entertain the bias that the more closely an IT employee works with users, the lower the level of the employee. (Nowhere would this principle be enunciated, but just look at the prestige of the roles in the organization and the salaries being paid.) Thus, a service desk agent is often considered to be an entry-level position. With time and experience, that agent might be “promoted” to the position of application administrator. But mere administrators cannot hold a candle to the geniuses that design and build the application!

An organization based on management discipline or process is well suited to this concept of career development. An organization based on the technology principle offers two avenues for career development. In a very few cases, movement from one team to another may be possible. There are affinities, for example, between database administration and directory services, or between desktop management and server management (depending on the operating systems in use). But there is no simple career path leading from network administration to software development.

Instead, organizations of this type generally promote people from engineer or administrator to team leader; from team leader to middle manager; from middle manager to senior manager, and so on. This is a perfect implementation of the Peter Principle.

When an organization is based on business domains, the situation is quite different. Within a single team, changing roles is as limited as in a technology-based organization. While such a development path is possible, it would require major retraining in many cases. The particular advantage of the business domain principle is that the retrained person already has knowledge of the business domain and how the team supports it. This concept reminds me of the adage I first heard forty years ago: “It is easier to teach programming to a chemist than chemistry to a programmer.”

The second type of development path involves retaining the same technical specialty, but moving to a team in a different business domain. In this way, an employee may develop over the years a comprehensive knowledge of the entire business and a good understanding of how all the business domains function together.  This same concept is theoretically possible for a process-based organization. However, in this latter case there is no simple way to gain the comprehensive business knowledge required. From the very start, the employee would be expected to serve all business domains for a given process or discipline. For this reason, a hybrid model, mixing discipline with business domain, is more feasible.

Standardization

Standardization is one of the benefits vaunted by the aficionados of the technology principle. They live in dread of the case where the best solution for a specific business domain depends on a technology that is not standard in the organization. I am not against standardization. I once worked for an organization that had no fewer than six different operating system versions on the desktop and every single desktop computer was created according to the models, operating systems and application versions currently available. I was able to bring that to exactly three standard hardware models and three different OS and application images, which had a gigantic impact on our ability to manage and support this infrastructure with the very limited resources available.

Standardization can mean two distinct things: the standardization of the technical service that a given technology team offers; and the standardization of the assets used to deliver those technical services. In fact, the one without the other is largely meaningless, is an arbitrary constraint.

For example, a storage service team might offer three types of disk storage differing according to throughput, RPO and overall volume of data. Each type of storage might have its own price and its associated levels of service. In principle, the team offers no other types of service, albeit a custom-solution could be devised at a much, much higher rate. In addition to segmenting the market for storage and providing the right qualities and service levels at the right prices for each segment (assuming the team has done a good job of analyzing that marketspace), standardization of the service leverages standardization of the fibre channel arrays, the volume manager software, the hierarchical storage systems, the disks, and so forth.

Similarly, a system administration team might have a standardized offer of platforms, based on the operating system, the processing capacity and the installed utilities and middleware. But this standardization is reasonable if the kit used to deliver each level of service is restricted to a very limited number of combinations. If the hardware (virtual or otherwise) were different each time, what point would there be in standardizing the services?

In addition to simplifying the complexities of management and support, standardization of assets can presumably result in lower per unit costs due to the additional buying power of the organization, as well as other forms of economies of scale.

As already mentioned, an organization based on management discipline or processes is apt to want to standardize the processes it uses. The assumption is that once standardized, the process is better suited for standardized measurement and for continual improvement. This assumption is based on extensive work done especially during the first half of the twentieth century on the optimization of manual labor. It was assumed, for example, that it was possible to find a single most effective and efficient way to dig a hole. All diggers working for the organization were expected to dig holes in that single, best way. This principle was extended to virtually all aspects of manual labor and industrial work.

But is the same true for the knowledge work that characterizes almost all the work done in IT? Is it really necessary for application developers to handle problems or incidents in precisely the same way as database administrators or network administrators? One can easily imagine that network administrators have a particular way of achieving the goals of incident management that is highly effective and efficient, but utterly unusable by application developers.

In short, a process-based organization might be effective in disseminating knowledge about a theoretical way to work or about a standard adopted in an organization, but there is no guarantee that this organization actually helps to better achieve the goals of those processes.

Standardization is hardly pertinent to organizations based on the business domain principle. IT trying to standardize the domains of a business is like a tail wagging the dog.

Which Paradigm is Best?

This brief discussion of the three paradigms and their various hybrid forms inevitably leads us to the question, “Which is best?” I am sure of only two points:

1. There is no single best organizational principle for all organizations.
2. We lack sufficient data to demonstrate the superiority of one paradigm over another, all other things being equal.

That being said, the astute reader will detect a certain disdain for the management discipline principle and a certain hesitation about favoring the technology paradigm. Given its strengths and weaknesses, what would be required to improve the suitability of the business domain paradigm for an organization? To answer this question, let us review the weaknesses of the paradigm and propose how to address them.

Technical Knowledge Sharing

If specialists are part of the same team, working with the same client base, perhaps sitting together in the same physical location, meeting informally and socially during which time they share experiences and knowledge, the usefulness of documented knowledge and any formal knowledge management is discounted. Julian E. Orr described this in his book Talking about Machines: An Ethnography of a Modern Job. (I am fully aware that this assertion is considered as heresy my many and I do not deny that some organizations have achieved useful, formal, highly structured management of knowledge. But I leave the pros and cons of this issue for another day.) How can useful knowledge about technology be identified, shared and used when the clientele is highly segmented and specialists do not typically sit together in the same workplace?

As Thomas H. Davenport has shown in Thinking for a Living,  while a laissez-faire attitude to knowledge is unlikely to deliver any improved results, incremental and agile approaches are more likely to deliver improvements than highly engineered, top-down approaches. This is especially true when work is of a highly collaborative nature, as opposed to being routine and transactional.

There are two opportunities in the cross-functional organization that could be encouraged as ways of sharing knowledge. The first takes advantage of the slack time that is inherent in a team where the specialist is not normally doing the work at a bottleneck. Slack time does not imply doing nothing at all. Instead, this time could be used for collaborative, pair-type work, with similar specialists in other teams. The important point is to provide face to face occasions where knowledge and information may be shared for practical purposes. The second method is to encourage socialization outside of work itself. The simplest method is to encourage lunches or coffee/tea breaks that bring together specialists.

In the worst case, specialists may share knowledge via technical communication means. When located at different sites, this may be the only practical way of sharing knowledge. But this would be true for teams organized on the technology principle, too. Depending on whether the culture of the organization encourages email, chats, telephony or other socio-technical means of communication, the important point is to give visibility to all specialists in the same field of the communications and to encourage all to monitor and respond to any requests for information or support.

Specialization and Effectiveness

Including specialists in a team where the average workload is significantly less than one FTE may be criticized in terms of wasting resources. So, how can we maintain all the skills required within a single team and ensure that each person is engaged for a decent amount of time? I see three possible solutions to this question, each one requiring a certain amount of compromising.

1. Share the specialist with other teams business domain teams

This “solution” is hardly a solution at all, insofar as it is a direct contradiction of the principle. A single person can only have ensured availability and minimized context switching if working for one team only. However, the following method of working might be acceptable.

Assume that every specialist plays a primary role within a certain team, as well as a secondary role for a different team. When, and only when, that person has slack time in the primary role, he or she might work as a “pair specialist” with a similar specialist in the other team.  Thus, the principle of “pair programming” is extended to other areas, such as system administration, but only when the primary role has slack time.

The advantage of this approach is that there are at least two people with direct and shared knowledge of each type of specialized work in each business domain. The problem of absent specialists is thereby mitigated, to a certain degree. Furthermore, certain risks of work, where four eyes might be better than two, could be reduced. Finally, a natural developmental path would be from secondary role in one team to the primary role in the other team.

This approach would have the disadvantage of increased context switching. However, there is probably a very big difference between switching contexts all the time among multiple tasks of changing priority and switching contexts only when the primary role has slack time.

2. Allow the specialist to perform enterprise-wide tasks during slack periods

Assume that all work items for the team have a higher priority than any other work in the organization. That other work should be performed during slack times, but the instant any team work items are ready for handling, the other work is dropped immediately and the team’s work is handled. What other type of work might be required? The two main areas would be in the areas of architectural standards and working on shared infrastructure projects.

There are two approaches to working on shared infrastructure projects, especially if the organizations is using a Kanban approach:

• The specialist works for only one team and all assigned work items appear on the Kanban board of that team. A higher level Kanban board is used to track the high level infrastructure project, but work items assigned to the specialist do not appear on that that high level Kanban board. In this case I assume that an architect or project manager is part of a team taking overall responsibility  for the infrastructure project.  No single business domain team could take the overall responsibility for such a project. The work of the project is broken down into manageable chunks, some of which are assigned to specialists from the business domain teams. In smaller organizations, where no such centralized project or architectural positions exist, this approach would be difficult to implement.
• The specialist works for two different teams, having assigned work items on two different Kanban boards. While theoretically possible, this approach doubles the communication costs for the specialists and removes the easy visibility of the work assigned to that specialist—assuming that physical, rather than virtual, Kanban boards are in use.

It would not be necessary for all specialists to perform these dual roles. Often, a specialist who might have skills adequate for a role within a certain team, be it a business domain or a technology team, but does not have the experience or skills to design architectures or shared infrastructures.

3. Create teams consisting of polyvalent or generalist members

The issue of the specialist having large amounts of slack time disappears if, instead, that person is a generalist, able to execute a wide variety of work items. Indeed, in smaller organizations that cannot afford to hire numerous persons but nonetheless need a wide variety of skills, this solution is not only possible—it is obligatory (unless the vast amount of work is outsourced). The question, then, is whether this approach is viable for larger organizations and if it could be part of hybrid, mid-sized organizations.

The open question is whether the benefits of specialization outweigh the benefits of responsibility for a broader range of activities in the life cycle of a product. Although this issue was already addressed by Adam Smith, several centuries ago, it is not clear that the principles established for manual labor are also applicable to knowledge work. People playing both specialist and generalists roles tend to weigh in vociferously in favor of one view or the other. They tend to have very strong biases based on their self-images and on the social roles they play with their organizations. While such biases must be understood and addressed when attempting to change from one organizational principal to another, this is not the question at hand. Research that compares these approaches in an objective way is required.

One might be tempted to say that the typical evolution from small start-up to large, successful organization generally follows the path from unstructured, temporary groups of generalists to highly structured organizational units of specialists. While this might have been true at the most general levels over the past century, it is precisely this question that needs to be re-examined in the light of knowledge versus manual work and in the light of a pull, limited work in progress approach to maximizing throughput to the customer, as opposed to a “traditional” approach of maximizing the busy-ness of each individual and of each activity.

The principle difficulty I see in this approach would be the difficulty in finding people with the needed mix of skills and experience. Many markets already find it very difficult to find the right single-skilled persons. Requiring a particular mix of multiple skills would only exacerbate this problem. Internal development programs might be a longer-term solution to this issue, so long as the organization is prepared to take the risk of investing heavily in personnel development, only to lose its personnel to other companies.

Standardization

The spectre that is likely to haunt organizations that are not structured on the technology principle is the lack of standardization. Standardization of technology may palliate may issues:

• allowing for lower per unit costs by negotiating prices for higher volumes
• simplifying interface requirements and architectures
• reducing technical constraints
• limiting the amount of technical knowledge required
• reducing the number of suppliers and contracts to manage.

If the constraints of standardization are to be an overall benefit to the organization, they must not unduly limit the fitness for purpose of the solutions provided. For example, it is obvious that if an off-the-shelf application can work equally well with four different database managers, there is no point is selecting a database manager that is not otherwise used in the organization. However, what about the case where an application that would provide the organization with a major competitive advantage, but that application works only with a database manager with which the organization has no previous experience or use? Insisting on standardization would be a case of biting off the nose to spite the face. In short, whatever the benefits of standardization, some means must be available for evolving the standards and for periodically assessing the possible trade-off between the gains and losses of the standards in place.

When an organization is based on the technology principle, there is a natural means for determining, applying and reassessing those standards. Can we obtain similar results when the business domain principle is applied? Once again, the work of standards definition and maintenance should be part of the background work done by specialists during their slack periods. An alternate way of thinking about these tasks that are offered at a lower priority than the direct needs of the business domain is to treat them as what David J. Anderson calls the “intangible” class of service (Kanban. Successful Evolutionary Change for Your Technology Business, p. 128f.)

But the organizational issue is not so much one of when the work gets done and with what priority. Rather, it is a question of how the work is organized and who performs it. Suppose there are eight system administrators in a company, each belonging to a different business domain team. No single team has the responsibility for defining and managing the standards for the computer systems in use. Furthermore, each system administrator is likely to have a different level of experience, knowledge and aptitude for doing the work of defining standards. How is the standardization work to be organized?

I can identify three options:

• the work is done via self-organization
• a specific technical job is defined for organizing this work
• the work is assigned to an architectural function in the organization.

When the work is self-organized, the specialists need to decide among themselves who is available to do the work, who has the skills to do the work and who emerges in a leadership role for the work. This agile and low overhead approach implies nonetheless that a certain amount of time is taken to perform such communications and organization. Each organization would need to determine pragmatically whether to do this work on demand (and what the triggers might be) or on a periodic basis, with the period determined and adjusted as required.

A second approach is to establish a set of technology jobs in the organization that are responsible for organizing all the infrastructure, technological, “intangible” work to be done. These jobs would essentially be product owner jobs for technologies, similar in nature to the product owner jobs that would feed requirements into the business domain teams. Although the product owner would not be a line manager, he or she would have a dotted line relationship with all the specialists in that particular technical product or technology. The technology product owner would need to understand the skills and experience of each specialist and would submit work items to the different business domain teams of which those specialists are members.

This approach would probably not be as well optimized for infrastructure work as the domain teams are optimized for business work. But this trade-off is probably beneficial overall, as it forces all priorities to be aligned with the business priorities of the organization. The infrastructure work will tend to be longer term work that is less sensitive to rapidly arising and changing, time-sensitive requirements. This is not to say that such work does not have deadlines. Rather, it understands that those deadlines are generally known very far in advance.

The third approach is very similar to the second approach, replacing individual technology product owners by architects. In fact, the architect would need to take responsibility from both an overall architectural perspective, as well as for the more detailed needs of individual technologies. The difference between the second and third approaches is largely a question of organizational size, wealth and maturity.

Full Organizational Integration

If we recognize the merits of and the potential in the business domain organizing principle, we should ask the questions, “Why limit this principle to IT? Why not organize the entire business in this way, with teams mixing together sales, marketing, operations, finance, personnel, IT and other employees?”

This question opens perspectives on a much larger subject, one that can hardly be addressed here. But I would add one thought. The rare IT employee who is a skilled manager has little chance to gain overall responsibility for operations or for research and development in a company. What a great pity. What is more, the vast majority of IT managers have been promoted to that role out of a technology-based role, and without any particular training in the areas of management or leadership. So we are faced with the problem of skilled IT people being promoted to roles for which they are not particularly qualified, while the same company might have skilled managers who never manage IT personnel, in the belief that technical knowledge is required.

Given the scarcity of good leaders and managers, can a company afford to misuse its people in this way? Why not benefit from company-wide integration of teams, benefiting from the best resources assigned to the most appropriate roles?

Why not, indeed?

The diagrams in this posting are licensed to you under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license.

Summary

Article Name

Organizing for Kanban

Description

What are the various organizational structures in IT and how do the relate to using kanban

Author

Robert S. Falkowitz

Publisher Name

Concentric Circle Consulting

Publisher Logo