多种应用 分发加速

应用交付服务

CRM ERP ... 各种核心数据库和应用

随时随地 安全可靠

GrandeNet - 互联网加速器

"伟大 (The Great)"的减速

由于许多国家和 ICP 供应商倾向于通过施加工具来控制或监视用户以达到优化他们“内部”数据访问的目的，致使整体网络状况恶化，性能明显下降。这个现状大大增加了为终端用户提供应用的公司的运营成本。我们的研究显示互联网的速度 — 不仅仅是在中国 — 在全球范围内都有着显著的差异并常常因为互联网没有选择优化路径而导致一些应用或一些用户短期或长期的访问缓慢。GrandeNet 可以通过使用 Re6st 在所有相连的服务器之间保持优质并稳定的连接路径，达到优化一些服务器80%的连接。

你会 Re6st 吗?

Re6st 是一个多协议随机网格生成器，使用 Babel 路由协议来发现网格中每个节点之间的最优路径。它可支持 IPv6 和 IPv4， 也即将实现 RINA 支持。它被商业化应用于 VIFIB － 分布式云供应商，用来解决因路由器上的漏洞，通过数据包检测来阻拦TCP协议，严格的政府信息过滤系统等造成的企业级分布式应用面临的互联网连接缺乏可靠性的问题。没有 re6st, 是不可能在一个分布式云上部署大型企业（如三菱， Sanef，Aide et Action等）所需的关键商业应用，也不可能在巴西、科特迪瓦，或者中国的部分缺乏可靠互联网环境的地方部署分布式云。

IPython Notebook

IPython Notebook 是一个基于网页的互动式计算环境, 用来创建嵌入 Python 代码的可执行 Notebooks。IPython Notebook 被研究人员大量用于研发并分享他们的科学成果。我们选择使用 IPython Notebook 在这篇文章中提供一个清晰的攻略。

这篇文章完全可以通过在你的 IPython Notebook 实例上导入这个 notebook 来重现。

为了执行这个 notebook，我们需要一些常见的 python 库，特别是 panda, numpy, scipy 和 matplotlib。以下是一些初始化库需要的导入。

In [1]:
%pylab inline
import sys
from matplotlib import pyplot
import matplotlib.pyplot as plt
#from mpl_toolkits.basemap import Basemap
from IPython.display import display, clear_output
from IPython.core.display import HTML 

import pandas as pd
import numpy as np

from pandas import rolling_median

np.set_printoptions(threshold="nan")

pd.set_option('display.max_rows', 2000)

from pandas import Series, DataFrame, Panel

Populating the interactive namespace from numpy and matplotlib

接下来，我们将定义核心代码 － 以方法的形式记录，将为这篇文章执行数据收集并计算结果。如果你对代码不感兴趣，你可以直接跳到这篇文章的下一章节。

In [2]:
import urllib2

def load_json(url, average="internet_ipv4", packet_lost="internet_ipv4_packet_lost"):
  """ Download JSON and normalize the data by:
        - Replace failed by a number '0', so the column is a float and not string
        - Replace 'average' and packet_lost by another name 
            for help concat 2 numpy array w/o recreate it.
        - Remove latest empty line.
  """
  req = urllib2.Request(url)
  response = urllib2.urlopen(req)
  content = response.read()
  return '[%s]' % content\
    .replace('"failed"', "0")\
    .replace('"average"', '"%s"' % average)\
    .replace('"packet_lost"', '"%s"' % packet_lost)\
    .replace("\n", ",")[:-1]

def load_test(id, date_id):
  """ Load Test results from Distributed Monitoring Tool
      and transform into DataFrames """
    
  # Load JSON for ICMPv6
  ping6_as_jsonstring = load_json(
    log_dict[id]["grandenet_ipv6"] % date_id, 
    average="grandenet_ipv6",  packet_lost="grandenet_ipv6_packet_lost")

  # Load JSON for ICMPv4  
  ping_as_jsonstring = load_json(
    log_dict[id]["internet_ipv4"] % date_id, 
    average="internet_ipv4", packet_lost="internet_ipv4_packet_lost")

  return pd.read_json(ping6_as_jsonstring, convert_dates=["time"]), \
    pd.read_json(ping_as_jsonstring, convert_dates=["time"])

def get_computer_list(dframeA, dframeB ):
    """ Extract all computer names at the DataFrames """
    return list(set([ computer_name[0] for computer_name in dframeA[["computer_name"]].as_matrix()] + 
                  [ computer_name[0] for computer_name in dframeB[["computer_name"]].as_matrix()]))

def get_computer_destination_label(dframeA):
    """ Determinate the Label Name for the computer which are receiving the ping"""
    return getComputerLabel([computer_name[0] for computer_name in dframeA[["name_or_ip"]].as_matrix()][0])

def getComputerLabel(computer_name):
    """ Translate hostname, ip addresses into meaningfull names for better understanting"""
    return server_label.get(computer_name, computer_name)

# Initiallization function which are going to be used for 
# collect the logs and transform them on Data Frames.
def plot_ping_comparation(df_ping6, df_ping):
  """ Function to load, plot and compare 2 Data Frames 
  """
  computer_list = get_computer_list(df_ping, df_ping6)

  computer_destination_label = get_computer_destination_label(df_ping6)

  measured_average = []
  packet_lost = []
    
  for computer_name in computer_list:
    
    if getComputerLabel(computer_name) == computer_destination_label:
      continue

    df6 = pd.DataFrame(df_ping6[df_ping6["computer_name"] == computer_name][df_ping6["grandenet_ipv6"] > 0][["time", "grandenet_ipv6"]])    
    df4 = pd.DataFrame(df_ping[df_ping["computer_name"] == computer_name][df_ping["internet_ipv4"] > 0][["time", "internet_ipv4"]])

    # Use Moving average in order to eliminate noise spikes on the chart and measurement.
    df6['grandenet_ipv6'] = rolling_median(df6['grandenet_ipv6'], window=3, center=True)
    df4['internet_ipv4'] = rolling_median(df4['internet_ipv4'], window=3, center=True)
    
    label = "'%s' to '%s'" % (getComputerLabel(computer_name), computer_destination_label)

    if 0 in [len(df6), len(df4)]:
      print "Found one empty array for %s" % label
      continue
    
    df = pd.DataFrame(pd.concat([df6, df4]))

    if SHOW_ALL_CHARTS:
      df4.plot(x="time", title=label + " (lower is better)", 
               sort_columns=["time"], figsize=(20,6))
      df6.plot(x="time", title=label + " (lower is better)", 
               sort_columns=["time"], color='r', figsize=(20,6))
    
    df.plot(x="time", title=label + " (lower is better)",
            marker='o', color=["b", "r"], figsize=(20,6))
    
    # Ignore 0 entries as it represents a full failure (so no average).
    ipv6_mean = df6["grandenet_ipv6"].mean()
    ipv4_mean = df4["internet_ipv4"].mean()

    grandenet_ipv6_packet_lost = df_ping6[df_ping6["computer_name"] == computer_name]["grandenet_ipv6_packet_lost"].mean()
    internet_ipv4_packet_lost = df_ping[df_ping["computer_name"] == computer_name]["internet_ipv4_packet_lost"].mean()
 
    if ipv6_mean < ipv4_mean:
      improvement_ratio = float(ipv4_mean - ipv6_mean)/ipv4_mean
      state = "OPTIMIZED in %sms (%.2f%%)" % ((ipv4_mean - ipv6_mean), improvement_ratio*100)
    elif ipv6_mean < (ipv4_mean + max(20, ipv4_mean*0.15)):
      state = "OK (in acceptable range %s < %s < %s)" % (ipv4_mean, ipv6_mean, (ipv4_mean + max(20, ipv4_mean*0.15)))
    else:
      state = "BAD (%sms slower)" % (ipv6_mean - ipv4_mean)

    measured_average.append({"name" : "'%s' to '%s'" % (getComputerLabel(computer_name), computer_destination_label),
                             "grandenet_ipv6": ipv6_mean,
                             "internet_ipv4": ipv4_mean,
                             "state": state})

    
    if grandenet_ipv6_packet_lost < internet_ipv4_packet_lost:
      loss_state = "OPTIMIZED (Better Packet Lost rate)"
    elif grandenet_ipv6_packet_lost == internet_ipv4_packet_lost:
      loss_state = "OK (Same Packet Lost rate)"
    elif (grandenet_ipv6_packet_lost - internet_ipv4_packet_lost) < 1:
      loss_state = "OK (less them 1% diference is considered same)"
    else:
      loss_state = "BAD (Worst Packet Lost rate)"

    packet_lost.append({"name" : "'%s' to '%s'" % (getComputerLabel(computer_name), computer_destination_label),
                             "grandenet_ipv6_packet_lost": grandenet_ipv6_packet_lost,
                             "internet_ipv4_packet_lost": internet_ipv4_packet_lost,
                             "state": loss_state})

  return pd.DataFrame(measured_average), pd.DataFrame(packet_lost)

用 SlapOS 分布式监测评估性能

GrandeNet 架构的核心是基于分布在多个云供应商 (亚马逊，青云，OVH，Rackspace，Ucloud 等) 服务器上以及分布在企业办公区或个人家庭的独立机器上。客户可以添加位于他们处所甚至是家庭的服务器，来用作主要的生产服务器。

这个 GrandeNet 混合动力和异构基础架构使用 SlapOS 管理和监测全球所有的分布式服务器。

在这篇文章中，我们使用一小组服务器 (12个) 使用 IPv4 运行 SlapOS 分布式监测。每个服务器都试图联系 (使用 ICMP 协议）使用 IPv4 和 IPv6 的12个服务器的地址。每10分钟运行10次测试 (10 pings) ，然后我们来测试并比较观察到的平均值和数据流失量。

下面这个图像说明了仅使用3个服务器时的测试:

GrandeNet Connectivity Example

下面我们初始化了每个服务器日志的位置，以及标签来提高图标和结果的可读性。

In [3]:
server_label = {
  'i-j0dshts2': "Guanghouz - Qincloud",
  '10-13-16-6': "Guanghouz - UCloud",
  'i-wbs0d67i' : "Hongkong - Qincloud 1",
  'i-vutfghrs': "Hongkong - Qincloud 0",
  'i-hf0f7ocn': "Beijing - Qincloud",
  'vps212661.ovh.net': "Strasbourg - OVH",
  'ip-172-31-30-97': "Singapour - Amazon",
  'ip-172-31-6-206': "Tokyo - Amazon",
  'ip-172-31-8-66' : "Virginia - Amazon",
  'ip-172-31-7-155': "US West - Amazon",
  'cloud-server-grandenet' : "Hongkong - Rackspace",
    
  'COMP-9': 'US West - Amazon',
  'COMP-8': 'Singapour - Amazon',
  'COMP-7': 'Tokyo - Amazon',
  'COMP-6': 'Hongkong - Qincloud 1',
  'COMP-4': 'Hongkong - Qincloud 0',
  'COMP-2': 'Beijing - Qincloud',
  'COMP-3': 'Guanghouz - Qincloud',
  'COMP-10': 'Guanghouz - UCloud',
  'COMP-11': 'Strasbourg - OVH',
  'COMP-12': 'Virginia - Amazon',
  "COMP-13": 'Beauharnois - OVH',

  "frontend0.grandenet.cn": "Beijing - Qincloud",
  "2401:5180::1": "Beijing - Qincloud",
  "frontend1.grandenet.cn": "Guanghouz - Qincloud",
  "2401:5180:0:6::1": "Guanghouz - Qincloud",
  "2401:5180:0:9::1" : "Hongkong - Qincloud 0",
  "frontend3.grandenet.cn" : "Hongkong - Qincloud 0",
  "2401:5180:0:8::1" : "Hongkong - Rackspace",
  "frontend4.grandenet.cn" : "Hongkong - Rackspace",
  "2401:5180:0:7::1": "Hongkong - Qincloud 1",
  "frontend5.grandenet.cn": "Hongkong - Qincloud 1",
  "2401:5180:0:c::1": "Tokyo - Amazon", 
  "frontend7.grandenet.cn": "Tokyo - Amazon",
  "2401:5180:0:d::1": "Singapour - Amazon",
  "frontend6.grandenet.cn": "Singapour - Amazon",
  "2401:5180:0:10::1": "US West - Amazon",
  "frontend8.grandenet.cn": "US West - Amazon",
  "2401:5180:0:13::1": "Guanghouz - UCloud",
  "frontend9.grandenet.cn": "Guanghouz - UCloud",
  "2401:5180:0:16::1": "Strasbourg - OVH",
  "frontend10.grandenet.cn": "Strasbourg - OVH",
  "2401:5180:0:15::1": "Virginia - Amazon",
  "frontend11.grandenet.cn": "Virginia - Amazon",
  "2401:5180:0:17::1": "Beauharnois - OVH",
  "frontend12.grandenet.cn": "Beauharnois - OVH",

}


log_dict = {
  "Hongkong - Qincloud 0": {
      "grandenet_ipv6": "https://softinst303.node.grandenet.cn/SOFTINST-314/ping6/log.%s.log",
      "internet_ipv4": "https://softinst303.node.grandenet.cn/SOFTINST-314/ping/log.%s.log",
      },
  "Virginia - Amazon": {
      "grandenet_ipv6": "https://softinst303.node.grandenet.cn/SOFTINST-322/ping6/log.%s.log",
      "internet_ipv4": "https://softinst303.node.grandenet.cn/SOFTINST-322/ping/log.%s.log",
      },
  "Strasbourg - OVH": {
      "grandenet_ipv6": "https://softinst303.node.grandenet.cn/SOFTINST-321/ping6/log.%s.log",
      "internet_ipv4": "https://softinst303.node.grandenet.cn/SOFTINST-321/ping/log.%s.log",
      },
  "Guanghouz - UCloud": {
      "grandenet_ipv6": "https://softinst303.node.grandenet.cn/SOFTINST-320/ping6/log.%s.log",
      "internet_ipv4": "https://softinst303.node.grandenet.cn/SOFTINST-320/ping/log.%s.log",
      },
  "Tokyo - Amazon": {
      "grandenet_ipv6": "https://softinst303.node.grandenet.cn/SOFTINST-317/ping6/log.%s.log",
      "internet_ipv4": "https://softinst303.node.grandenet.cn/SOFTINST-317/ping/log.%s.log",
      },
  "US West - Amazon": {
      "grandenet_ipv6": "https://softinst303.node.grandenet.cn/SOFTINST-319/ping6/log.%s.log",
      "internet_ipv4": "https://softinst303.node.grandenet.cn/SOFTINST-319/ping/log.%s.log",
      },
  "Singapour - Amazon": {
      "grandenet_ipv6": "https://softinst303.node.grandenet.cn/SOFTINST-318/ping6/log.%s.log",
      "internet_ipv4": "https://softinst303.node.grandenet.cn/SOFTINST-318/ping/log.%s.log",
      },
  "Hongkong - Qincloud 1": {
      "grandenet_ipv6": "https://softinst303.node.grandenet.cn/SOFTINST-316/ping6/log.%s.log",
      "internet_ipv4": "https://softinst303.node.grandenet.cn/SOFTINST-316/ping/log.%s.log",
      },
  "Guanghouz - Qincloud": {
      "grandenet_ipv6": "https://softinst303.node.grandenet.cn/SOFTINST-313/ping6/log.%s.log",
      "internet_ipv4": "https://softinst303.node.grandenet.cn/SOFTINST-313/ping/log.%s.log",
      },
  "Beijing - Qincloud": {
      "grandenet_ipv6": "https://softinst303.node.grandenet.cn/SOFTINST-312/ping6/log.%s.log",
      "internet_ipv4": "https://softinst303.node.grandenet.cn/SOFTINST-312/ping/log.%s.log",
      },
  "Hongkong - Rackspace": {
      "grandenet_ipv6": "https://softinst303.node.grandenet.cn/SOFTINST-315/ping6/log.%s.log",
      "internet_ipv4": "https://softinst303.node.grandenet.cn/SOFTINST-315/ping/log.%s.log",
      },
  "Beauharnois - OVH": {
      "grandenet_ipv6": "https://softinst303.node.grandenet.cn/SOFTINST-625/ping6/log.%s.log",
      "internet_ipv4": "https://softinst303.node.grandenet.cn/SOFTINST-625/ping/log.%s.log",
      }
}

我们同时还限制了这篇文章的规模，使测试运行在一定日期范围内，见如下变量“DAY”。

In [4]:
# Define here if you want more or less charts verbosity. Show all charts can
# make this report quite big.
SHOW_ALL_CHARTS = False

# Generate Report for the Jan, 28, 2016
DAY = "20160128"

从分布式 SlapOS 监测收集数据

为了出结果，我们使用了如上定义的方法，抓取日志并转化为数据帧。这些数据帧包含了指示时期 (DAY) 的测试结果。

In [5]:
hq0_df_ping6, hq0_df_ping = load_test(id = "Hongkong - Qincloud 0", date_id=DAY)
In [6]:
va_df_ping6, va_df_ping = load_test(id = "Virginia - Amazon", date_id=DAY)
In [7]:
gu_df_ping6, gu_df_ping = load_test(id =  "Guanghouz - UCloud", date_id=DAY)
In [8]:
sa_df_ping6, sa_df_ping = load_test(id = "Singapour - Amazon", date_id=DAY)
In [9]:
hq1_df_ping6, hq1_df_ping = load_test(id = "Hongkong - Qincloud 1", date_id=DAY)
In [10]:
hr_df_ping6, hr_df_ping = load_test(id = "Hongkong - Rackspace", date_id=DAY)
In [11]:
wa_df_ping6, wa_df_ping = load_test(id = "US West - Amazon", date_id=DAY)
In [12]:
go_df_ping6, go_df_ping = load_test(id = "Strasbourg - OVH", date_id=DAY)
In [13]:
ta_df_ping6, ta_df_ping = load_test(id = "Tokyo - Amazon", date_id=DAY)
In [14]:
gq_df_ping6, gq_df_ping = load_test(id = "Guanghouz - Qincloud", date_id=DAY)
In [15]:
bq_df_ping6, bq_df_ping = load_test(id = "Beijing - Qincloud", date_id=DAY)
In [16]:
bho_df_ping6, bho_df_ping = load_test(id = "Beauharnois - OVH", date_id=DAY)

互联网 IPv4 vs Grandenet IPv6

使用数据帧，我们可以视觉化在使用互联网 IPv4 (红色）和 Grandenet IPv6 (蓝色) 时的响应时间（毫秒）的比较。 由于我们使用的是 ICMP 协议来测量响应时间，下面的图表使用了“ping"作为 IPv4的名字，ping6 作为 IPv6 的名字，并且突出了互联网 IPv4 和 IPv6 之间的区别。响应时间越短， 绘制线越低，越好。

In [17]:
hq0_average_dataframe, hq0_packetloss_dataframe = plot_ping_comparation(hq0_df_ping6, hq0_df_ping)

/srv/slapgrid/slappart8/srv/runner/software/9a8d67b31671ba36ac107c65a141c073/develop-eggs/pandas-0.16.2-py2.7-linux-x86_64.egg/pandas/core/frame.py:1825: UserWarning: Boolean Series key will be reindexed to match DataFrame index.
  "DataFrame index.", UserWarning)
In [18]:
va_average_dataframe, va_packetloss_dataframe = plot_ping_comparation(va_df_ping6, va_df_ping)
In [19]:
go_average_dataframe, go_packetloss_dataframe = plot_ping_comparation(go_df_ping6, go_df_ping)