![]() Imagine you want to compare the hourly visitor count in New York City’s Central Park with the shelter-in-place order in effect and before it. The two-sided Kolmogorov–Smirnov test calculates a probability that two samples are from the same underlying distribution. ![]() We can apply the Kolmogorov–Smirnov test between distributions and calculate a p-value. Distribution of tests per hour, by protocol.Īt first glance, it’s quite difficult to draw any conclusions, except that each protocol was tested at least once every hour. ![]() ![]() As test combinations were randomly selected every two minutes, 24 hours a day, intuition says that their distribution should be even. To avoid falling into such a statistical nightmare, observations of each test combination should be distributed across 24 hours as evenly as possible. You can do it, but it doesn’t tell you anything useful. Comparing the results of these tests would be the same as comparing the auto-brightness of your laptop screen during the day and during the night. Why? Let’s say you run a test for one protocol at 9 pm when the internet usage at your selected time zone is peaking, and run a test for another protocol at 4 am when the internet usage is at its lowest. To make a valid comparison, we need to make sure to equalize environmental conditions. We can start with the timing of the tests. Let’s take a deep dive into the confidence of our speed testing results and see if we can tick all the boxes on accurate measurement. Yet, one question is still left unanswered: how many tests should you take to be certain that one protocol is faster than another? We covered quite a few reasons for this in the previous post, where we also explained the methodology of the extensive testing we did to measure NordLynx performance. While most of the time NordLynx outperformed other protocols, there were some cases with slightly worse speed results.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |