...

Whether you used code or a set of arbitrarily split links (first name A-M, first name N-Z, etc.), you can change this partway through the test to even up the numbers. For example, you may change the split so that test A now gets 30%

• If you used code, you can change it from a 50/50 split to 80/20 in favor of your under-supplied group.
• If you used a set of links, you can change the split from something like "first name A-M, first name N-Z" to "first name A-E, first name F-Z" so that the first test now gets 20% of the clicks, while the second test

...

• (the one that’s lagging) gets

...

• 80%.

...

• Some tasks may be confusing or misleading.
  This is especially likely if you didn’t properly pilot your test. Some tasks are hard to phrase clearly without giving away the answer, but remember that a confusing task is a problem in your study, not necessarily a problem in the tree itself. You shouldn’t change the wording during the test, but you should revise in your next round of testing.
  
  
• Some correct answers aren’t marked as “correct”.
  After doing hundreds of tree tests, we still run into this wrinkle all the time. When we set up each task, we try to mark all the correct answers for it. However, in a large tree, each task may have several correct answers, and it’s likely we’ll miss a few.
  Because of this, a good testing tool should let you (as the test administrator) change which answers are correct for each task, either while the test is running or afterward when you’re doing your analysis. We often find that test scores go up substantially when we do this post-test correct. For more on this, see Chapter ~, Analyzing Results Cleaning the data in Chapter 12.
  
  
  

Ideally, a high task score means that you did your job well when you created the tree.

...