3
$\begingroup$

Reading the Microsoft TechNet article "Test results: Extra-large scenario (FAST Search Server 2010 for SharePoint)", I came across this graph.

enter image description here

Now, am I stupid (and if so, could someone please enlighten me), or is this graph just stupid? To me, it seems like it's created with MS Paint instead of reflecting real values - I can not make sense of the graph as it seems like any given value on the horizontal axis (Queries per second) can result in multiple values on the vertical axis (Latency).

EDIT: Thanks to all who contributed, the curves (not graphs) make more sense to me now. My conclusion to the original question is somewhere in the middle. I admit some stupidity on my own behalf, but still think the graphical representation of the data is quite stupid, as it fails to clearly communicate what it should.

  • 0
    @Christian: I sincerely doubt that it is a question of how the graphs were *painted*. This rather has to do with what the raw data was, and what the graph is $m$eant to represent. I doubt it's supposed to be a single-valued function simply because latency does not depend exclusively and solely on the number of queries per second; that is, this is not the graph of a *function*, but rather this is a *curve*.2011-06-20

2 Answers 2

3

The only mathematics here is whether these curves represent functions, which they do not. They represent data taken as a function of time. these curves just show the behavior of the system with increasing time.

  • 0
    The way reason they chose a line instead of points is so that you can "follow" along and see the evolution of the system. Otherwise you would come to a point where there are two y-values for an x value and you would not know what that meant. Again, think of time evolution. If you want to know why the graphs curve backwards, from article," the farm was able to sustain about 15 queries per second (QPS) with less than 1 second average latency before being limited by the CPU resources on the search row. During incremental crawls and full crawls, this number dropped to 12.5 and 10 QPS respectively"2011-06-20
1

Reading the article you mentioned, it seems to me that they did multiple runs of the same question, which is why you get multiple values for a given number of queries per second. What seems very unorthodox is that they polished that by drawing the lines which seems to reflect the timing of the measures (you get sort of a dynamic trajectory of the system).

  • 0
    Sounds reasonable.2011-06-20