Notices of the American Mathematical Society

Welcome to the current issue of the Notices of the American Mathematical Society.
With support from AMS membership, we are pleased to share the journal with the global mathematical community.


Ability and Diversity of Skills

William Geller
Michał Misiurewicz

Communicated by Notices Associate Editor William McCallum

1. Introduction

The aim of this paper is to build a simple model of problem solving, both by single agents and by teams. We realize that our model is crude and of course far from universal. Yet the results we get seem to us quite illuminating, and show the importance of both ability and diversity of skills.

The question of how to measure effectiveness of problem solving by individuals and (even more importantly) by teams, and how to choose the best individual/team, has been a subject of a lot of research. We can give as examples papers GJIBHP2KIKR, and the literature cited there. We do not address explicitly the problem of choosing a team, but our findings may serve as the basis for further research in that direction (in cases where it seems that our model may be applicable).

For a single agent, or a team of agents, we try to measure the probability of success as a function of the difficulty of a problem (or rather the easiness of the problem, measured by a variable ; the larger , the easier the problem). In Section 3 we show that the probability of success is concave as a function of .

In Section 4 we show that in our model for a single agent specialization is better than versatility.⁠Footnote1 We also show that comparing agents is difficult. In most cases, for a given agent and chosen values of , there can be another agent, who is better at solving problems with easiness for those chosen values, but worse at solving problems with all other values of .

1

Thus a strategic agent might choose to specialize.

In Section 5 we consider teams of agents. We get what can be considered the main result of the paper: whenever our model can be applied, both abilities of the team members and the diversity of skills in the team matter. If any of those increases, so does the probability of success. For simplicity, we consider teams with two members, but it is clear that similar results should hold for larger teams. Interestingly, there is an example where for easier problems ability is more important, but for more difficult problems diversity is more important.

In Section 6 we show how our model can be applied to a situation where the agents are trying to defend an organization against an attack. In this application, diversity is even more important than for general problem solving.

Some of our ideas came from studying the model of L. Hong and S.E. Page HP1. Our model is much simpler, and can be easily investigated both by pure mathematical means and by computational means. Moreover, we avoid the main deficiency of the Hong–Page model, where high-ability teams consist basically of clones of the same agent (and as a result, ability excludes diversity).⁠Footnote2

2

A reader interested in the discussion about that model may want to look at T1PT2.

2. Preliminary Model

If an agent will be trying to solve problems that are not known in advance, her expected performance can be measured by an average over various possible problems.

Our first, preliminary model is as follows. An agent has some set of skills. This set is a subset of . An immediate problem is represented by a subset of of cardinality . An agent can make progress if the intersection is nonempty. Clearly, the difficulty of the problem is measured by ; problems with smaller are more difficult.

When we want to measure the ability of an agent, we average performance of an agent over all problems of a given difficulty (that is, with a given cardinality ). The result clearly does not depend on a concrete set of skills, but only on its cardinality (the skillfulness of the agent).

For given it is easy to compute the probability of making progress. If , this probability is . If , out of all possible sets only result in failure. Therefore the probability of success (that is, making progress) is

In Figure 1, we can see how the probability of success varies with the difficulty of the problem, for agents with various numbers of skills. If the problem is easy, the skillfulness of an agent does not matter much (provided the agent has some minimal number of skills). However, for difficult problems it matters a lot.

Figure 1.

For , graphs of the probabilities of success as functions of the difficulty of the problem for various numbers of skills of the agent. As we move to the right, the difficulty of the problem decreases (that is, increases).

Graphic without alt text

3. Main Model

The preliminary model is very crude, because for each skill an agent either has it or does not. However, one should allow an agent to have partial skills. Then becomes a strength function . If , then the probability that the agent can make progress using skill number is . We assume that those probabilities for different are independent. This means that it is easier to use in computations the weakness function , where is the constant function 1. Then the probability of failure for a given agent and given problem is equal to the product of the numbers over .

Often instead of speaking of the strength and weakness functions we will speak of the strength and weakness vectors and .

For a given , the sum of over all sets of cardinality is equal to

where denotes the cardinality of . Observe that this number is equal to the coefficient for the polynomial

of . Thus, the average probability of failure over all sets of cardinality is equal to this coefficient divided by . Note that is the coefficient of for the polynomial . This in particular means that if an agent has no skills (so ), her probability of failure is 1 no matter what.

Clearly, if is a permutation of the set then . Therefore we may assume that . Sometimes, if we do not want to make this assumption, we will say that is a permutation of .

Of course, if takes only values 0 and 1, we get the previous model.

Let us investigate some basic properties of the function .

Proposition 1.

We have , and equality holds only if either both numbers are equal to or .

Proof.

Replace each subset of cardinality by pairs , where . Then the average of over all such pairs will be equal to . Similarly, when we replace each subset of cardinality by pairs , where , the average of over all such pairs will be equal to . However, there is a natural one-to-one correspondence between the pairs of the first and of the second type. Namely, if and , then and . Since always , we get .

Suppose that we have the equality. Then for every of cardinality and every we have either or . If for every of cardinality we have , then . Otherwise, there exists of cardinality with , so we have for every . Unless , there is for which . Choose one and consider the set . Then

a contradiction. This proves the second part of the proposition.

Proposition 2.

We have

so the function is convex (and the success function, , is concave). Equality holds if and only if either or there is at most one such that .

Proof.

In a similar way as in the proof of Proposition 1, we get the following four equalities (in the first one, we have to look at the set instead of ):

Since , we have

Now, from 5, 4, and 6, we get

From 2 and 3, we get

From 7 and 8 (note that 8 holds also with and switched) we get

that is,

Since

we get

This proves the first part of the proposition.

To prove the second part, notice that by 7, equality in 1 holds if and only if for every of cardinality and every such that , either , or , or .

If , then for every of cardinality we have . If there is at most one such that , then implies or . In all those cases we get equality in 1.

Now assume that and for at least two indices . Then there are two possible cases. Either there are two or more zeros among , , or there is at most one zero there. In the first case, we can choose such that and . Then, since , there is of cardinality , such that . In the second case, we choose with and as small as possible, and again there is of cardinality , such that . In both cases, there is no equality in 1.

4. Specialization and Versatility

We would like to be able to measure the skillfulness of an agent in our model. There may be various ways of doing this, and as we will see in Theorem 5, we cannot expect to find a perfect one. We will settle on what seems the most natural way of doing it, by defining it to be , where is the strength function of the agent.

Within our model, one of the first questions that comes to mind is what is the best distribution of strengths given the skillfulness of an agent. The agent can be more specialized or more versatile. We will show that in our model specialization is better than versatility.

The simplest case is when we have two agents, one with and for , and the other one with and for . The first agent is more versatile and the second one more specialized. We have

and

Thus, for , and for . Let us make exact computations.

The coefficient of for the polynomial is the same as the coefficient of for the polynomial , that is, . Thus,

To get the coefficient of for the polynomial , we have additionally to add the coefficient of for the polynomial , so

This means that while the graph of the probability of success as a function of lies on the straight line from to for , it lies on a parabola from to for (see Figure 2).

In this example specialization is better than versatility (see Figures 3 and 4 for other examples).

Figure 2.

The same picture as in Figure 1, with additional red graphs showing probabilities of success with skillfulness an integer, from 1 to 10, but taking only values 0 and . Black graphs represent more specialization and red ones more versatility.

Graphic without alt text
Figure 3.

The same picture as in Figure 2, but the red graphs showing probabilities of success with skillfulness an integer, from 1 to 20, spread evenly (that is, is the same for all ). This represents even more versatility than in the preceding figure.

Graphic without alt text
Figure 4.

Here skillfulness is 1, but it is spread equally into skills, and varies from 1 (the highest graph) to 20 (the lowest graph).

Graphic without alt text

We considered only a simple example, but it turns out that in more complicated situations the result is the same.

Lemma 3.

Let , . Let a strength function be such that and . If , set , and for . If , set , , and for . Then, in both cases, for all and for at least one .

Proof.

Assume first that . Then for some polynomial of degree with nonnegative coefficients we have

and

Thus, for all , and for at least one .

Assume now that . Then the formula for stays the same, and we have

We have , so the result is the same as in the first case.

The way we can restate this lemma is that if an agent has at least two strengths other than 0 and 1, then we can change her strength function, keeping the same skillfulness, in such a way that none of the probabilities of failure increases, and at least one of them strictly decreases. This change of the strength function is in the direction of specialization.

Observe that given a skillfulness , there is a unique (up to permutations) strength function with at most one value in giving this skillfulness. Let us denote this function by . This is the strength function of the most specialized possible agent of skillfulness .

Theorem 4.

Given a strength function with skillfulness and probabilities of failure , and the function with probabilities of failure , we have for every , and unless is a permutation of , there is at least one for which .

Proof.

Use Lemma 3 inductively.

We can interpret this result as saying that for an individual problem solver in our model, specialization is better than versatility.

In Figures 2 and 3 we see pairs of graphs of the probability of success (extended piecewise linearly to functions on ) that have intersections not only at 0 (more such graphs can be seen in Figures 6 and 8). For each such pair there is only one intersection apart from 0. Thus, a question arises whether this is a general phenomenon. We now show that this is very far from being true.

Theorem 5.

Let be an injection, and let be a function from to . Then there exists a function such that if and if .

Proof.

We may assume that is a strictly increasing function. We have , where

If is sufficiently small, then the zeros of the polynomial are all real and contained in the interval . Thus, there is a function such that this polynomial is equal to . We have , so if and if .

This theorem illustrates the difficulty of measuring the ability of agents (see also, for example, HP2KR). Theorem 5 shows that given a typical agent and a specified set of problem difficulties, there can be another agent who is better at solving problems with those difficulties but worse at solving problems with all other difficulties.

One can ask whether we can remove the assumptions that maps to the open interval and that it is an injection. The answer is “no,” as the following simple examples show.

Let and consider two weakness vectors, and . We have and . If then , so

Thus, we cannot have and .

Similarly, if and , then and . If then by the inequality between the geometric and arithmetic means, , so . Thus, we cannot have and .

5. Teams

Let us consider now what our model tells us about teams of agents. Suppose we have a team of two agents,⁠Footnote3 and for skill , their weakness is and , respectively. We assume independence (in the sense of probability theory), so the probability of not making progress on a problem using skill is the product . As for a single agent, we will speak of the strength and weakness vectors of a team.

3

The situation should be similar for larger teams.

We can take the diversity of a team to be the lack of overlap of their strengths. While this is not a formal definition, we can often say which of two teams has larger diversity. Similarly, we can speak of the ability of the team. Here we can use the skillfulness as the measure, although Theorem 5 suggests that it is not an ideal measure. However, again we can often say which of two teams (or members of the team) has larger ability.

Let us consider the simple example where there are two agents in the team, and each of them has two skills of strength . There are three possibilities: two, one, or none of the skills coincide. Then we get for the team three possible strength vectors: , , and . Figure 5 illustrates the results. We see that with the same levels of abilities of the members of the team, more diversity in their skills gives better results, except for the easiest problems.

Figure 5.

Team of two agents, each with two skills of strength 1/2. The black graph corresponds to a team whose skills coincide, the red graph to a team sharing one skill, and the green one to a team with no skills in common.

Graphic without alt text
Figure 6.

Like Figure 5, but with two additional graphs. The dark blue graph corresponds to the team strength vector (that is, strength of skills , but no diversity), the orange graph to (that is, strength of skills and maximal diversity).

Graphic without alt text

This phenomenon is easy to explain. Consider two vectors and , the same except the values at some . They come from a team of agents with prescribed skills, slightly differently placed. For we have more diversity, so comes from one agent, and from another agent. For , diversity is smaller, so and . Then there is a polynomial T, with nonnegative coefficients, such that and . We have and and , the coefficients of the polynomial are strictly larger than the coefficients of the polynomial (except the first and last coefficients, for which we have the equality). This means that if ability is kept constant, by increasing diversity of skills we get better chances for success.

This result differs from what we saw about specialization and versatility. This is because the skillfulness of a team is usually smaller than the sum of each member’s skillfulness.

We can ask what happens if we change the abilities of the members of the team. In Figure 6 we added two graphs. One of them corresponds to larger abilities but no diversity; the other one corresponds to smaller abilities but larger diversity. By comparing the two lowest graphs with each other, and two highest graphs with each other, we see that to some degree ability and diversity of skills are exchangeable. However, in this example, for easier problems ability is more important, while for more difficult ones diversity is more important. Of course, we do not know how this applies to real life situations, since our model may or may not fit them (cf. Theorem 5).

However, if the ability of one or more team members increases (and no other changes are made), the coefficients of the polynomial for the team decrease, so we get better chances for success for problems of all difficulties.

6. Security

Let us look at a possible adaptation of our model to a security problem. Here the agents are trying to defend an organization against an attack (for instance, by hackers).

Figure 7.

Team of two agents for the security problem. The black graph corresponds to the strength vector consisting of ten strengths and ten 0; the red graph to the vector consisting of five strengths , ten , and five 0; and the green one to the vector consisting of twenty strengths .

Graphic without alt text

The attacker has possible lines of attack (out of possible), and for each of them the skillfulness of the agent gives us the probability of stopping this line of attack. Thus, the average probability of stopping an attack of strength is . Note that for problem solving large meant an easy problem; here large means a strong attack.

In earlier sections we wanted to minimize our probability of failure . In contrast, we want now to maximize the attacker’s chance of failure . Therefore, by the results for problem solving, for security purposes versatility is better than specialization.

This also means that in this application diversity of skills in a team plays an even larger role than for problem solving. Diversity corresponds to more uniform spread of strengths, which for the security problem is useful even for one agent. An example similar to the one from Figure 5 is illustrated in Figure 7. We consider a team consisting of two agents, each of them having 10 strengths . Then we compare three possibilities: all, half, or none of the strengths coincide. We get for the team three strength vectors. The first one consists of ten strengths and ten 0; the second one of five strengths , ten , and five 0; and the third one of twenty strengths .

Figure 8.

Team of two agents for the security problem. The red graph corresponds to the vector consisting of twenty strengths ; and the blue one to the vector consisting of four strengths , twelve strengths , and four strengths .

Graphic without alt text

Here also diversity and ability are to some degree interchangeable. For example, if we have two agents, one with sixteen strengths , and the other one with four strengths , then in the most diverse case we get for the team twenty strengths . If the first agent’s strengths are instead of and we consider the least diverse case, we get for the team four strengths , twelve strengths , and four strengths 0. The first team will be better for strong attacks, but the second one will be better for weak attacks (see Figure 8).

References

[GJIB]
S. M. Gully, A. Joshi, K. A. Incalcaterra, and J. M. Beaubien (2002), A meta-analysis of team-efficacy, potency, and performance: Interdependence and level of analysis as moderators of observed relationships, Journal of Applied Psychology 87 (5), 819-832.
[HP1]
L. Hong and S. E. Page (2004), Groups of diverse problem solvers can outperform groups of high-ability problem solvers, Proc. Nat. Acad. Sci. 101 (46), 16385-16389.
[HP2]
L. Hong and S. E. Page (2021) Does a Test Exist? On the Possibility of Individual Hiring Criteria for Optimal Team Composition, available at SSRN: https://ssrn.com/abstract=4035941 or http://dx.doi.org/10.2139/ssrn.4035941
[KI]
S. W. Kozlowski and D. R. Ilgen (2006), Enhancing the effectiveness of work groups and teams, Psychological Science in the Public Interest 7 (3), 77-124.
[KR]
J. Kleinberg and M. Raghu (2015), Team Performance with Test Scores, Proceedings of the 16th ACM Conference on Economics and Computation, 511-528.
[P]
S.E. Page (2015), Letter to the Editor, Notices Amer. Math. Soc. 62 (1), 9-10.
[T1]
Abigail Thompson, Does diversity trump ability? An example of the misuse of mathematics in the social sciences, Notices Amer. Math. Soc. 61 (2014), no. 9, 1024–1030, DOI 10.1090/noti1163. MR3241558Show rawAMSref\bib{T1}{article}{ label={T1}, author={Thompson, Abigail}, title={Does diversity trump ability? An example of the misuse of mathematics in the social sciences}, journal={Notices Amer. Math. Soc.}, volume={61}, date={2014}, number={9}, pages={1024--1030}, issn={0002-9920}, review={\MR {3241558}}, doi={10.1090/noti1163}, } Close amsref.
[T2]
A. Thompson (2015), Letter to the Editor, Notices Amer. Math. Soc. 62 (1), 10.

Credits

All figures are courtesy of Michał Misiurewicz.

Author photos are courtesy of the authors.