Just can't get enough

I just can’t get enough!   It is a familiar cry from infrastructure managers the world over.  No matter how much resources you throw at an application or service, the infrastructure manager will claim that they still don’t have enough resources to get a good quality of service from the application.

We Capacity Managers know that the solution isn’t as simple as just throwing more CPU or more Memory at the problem (although that’ll be what is being asked for).  The client might equally well be asking for more disk space, or maybe more disk spindles, or maybe more network bandwidth.  Either way… whatever they have, it won’t be enough.

Recently, this perpetual insatiable demand for “more”, has been observed in a subtly different way.

I have been managing a client’s private cloud as referred to many times in past blogs.  The client allowed users to select the configuration of their guests from a pre-defined pick-list of sizes.

As you can see, there are only 7 size options, and there is a fixed ratio of 4Gb of vMemory for each vCore.  This type of tightly defined shopping cart is very useful to the Capacity Manager.  We know that there can only be a limited number of building-blocks for allocation within the Cloud.  Furthermore, we can monitor the existing client behaviour and work out the probability of each option (A-G) being chosen.  This gives us a typical distribution of choices as shown in the chart below.  We can take this a step further, and work out how many guests in a typical option distribution we can fit onto a typical cluster.

Any high school student knows about the Normal distribution.  They would expect that the majority of users would select guests of size C, D, and E.  Only a very few users would choose option A (the smallest) or option G (the largest).  However the chart shows something different.

More people that otherwise expected are choosing the larger guest sizes.  Even though these options cost sufficiently more that the smaller options, human nature comes into play.  People choose the larger sizes simply because they can.    This was brought into sharp focus when the shopping cart of options above was changed.

The client decided that, rather than fixing the ratio of vMemory to vCore at 4:1, they would release the shackles and allow clients a freer rein of choices.

As you can see, while the choices of vCores remained unchanged, the client introduced two new options for vMemory (24 and 256).  The impact on the distribution of selected vCores and vMemory is shown in the two charts below.  Again, more users than would normally be expected are selecting the maximum vCore and vMemory possible, and (although I won't go into the detailed calculation here) readers with experience in mathematics will immediately notice that the more pronounced increase in vMemory requested compared to vCores means that the traditional 1:4 ratio of vCore to vMemory has shifted to closer to 1:5.

In the client’s private cloud architecture (where resource limits are based upon Allocation rather than Utilisation as blogged about earlier) this further reduced the number of clients that can be accommodated within each cluster.  When interviewing users about the choices that they made, they clarified that they had ordered the larger sizes simply “because we can”.  Even when the impact of this choice on their fellow users was explained (ie: that by ordering the largest sizes of guest it meant that fewer users would be able to build their guests) user behaviour was unchanged.

What this showed me, is that there is a part of the Capacity Manager’s role which has to include considering the psychology of users.  It is human nature to a) be selfish, and b) to want the best you can get.  It is possible to teach sharing and consideration for others.. however in a competitive business environment you will be fighting a losing battle.  You cannot consider technical decisions in isolation without considering how human psychology will change that into an actual outcome.

Users…. They just can’t get enough!!!