graphite安装(一键搞定版)

  • Post author:
  • Post category:其他


一、python部分:

0、系统:

$ cat /etc/redhat-release 
CentOS Linux release 7.2.1511 (Core) 

1、检查系统python是否2.7以上(最新graphite需要python至少2.7):

$ python -V
Python 2.7.5

2、安装pip:

yum install python-pip
$ pip -V
pip 8.1.2 from /usr/lib/python2.7/site-packages (python 2.7)

这里安装后的版本是8,后面还需要再升级一下pip,因为安装cairocffi时需要。

二、下载相关源码、并且安装:

1、git下载graphite相关源码下载、安装:

cd /opt
下载源码:
git clone https://github.com/graphite-project/graphite-web.git
git clone https://github.com/graphite-project/carbon.git
git clone https://github.com/graphite-project/whisper.git

然后通过python setup.py install安装:
cd /opt/graphite-web 
python setup.py install 
cd carbon
python setup.py install
cd whisper
python setup.py install

这时在/opt下多了一个graphite目录,/opt目录如下

drwxr-xr-x  9 root root 4096 Sep 17 10:30 carbon
drwxr-xr-x  9 root root   92 Sep 17 11:22 graphite
drwxr-xr-x 11 root root 4096 Sep 17 10:29 graphite-web
drwxr-xr-x  3 root root   19 Sep 17 09:32 soft
drwxr-xr-x  6 root root 4096 Sep 17 10:30 whisper

2、检查依赖:

cd /opt/graphite-web
$ python check-dependencies.py
[REQUIRED] Unable to import the 'cairocffi' module, attempting to fall back to pycairo
[REQUIRED] Unable to import the 'cairo' module, do you have pycairo installed for python 2?
[REQUIRED] Unable to import the 'django' module, do you have Django installed for python 2?
[REQUIRED] Unable to import the 'pytz' module, do you have pytz module installed for python 2?
[REQUIRED] Unable to import the 'pyparsing' module, do you have pyparsing module installed for python 2?
[REQUIRED] Unable to import the 'tagging' module, do you have django-tagging installed for python 2?
[OPTIONAL] Unable to import the 'memcache' module, do you have python-memcached installed for python 2? This feature is not required but greatly improves performance.
[OPTIONAL] Unable to import the 'ldap' module, do you have python-ldap installed for python 2? Without python-ldap, you will not be able to use LDAP authentication in the graphite webapp.
[OPTIONAL] Unable to import the 'txamqp' module, this is required if you want to use AMQP as an input to Carbon. Note that txamqp requires python 2.5 or greater.
[OPTIONAL] Unable to import the 'python-rrdtool' module, this is required for reading RRD.
[OPTIONAL] Unable to import the 'whitenoise' module. This is useful for serving static files.
[OPTIONAL] Unable to import the 'pyhash' module. This is useful for fnv1_ch hashing support.
6 optional dependencies not met. Please consider the optional items before proceeding.
5 necessary dependencies not met. Graphite will not function until these dependencies are fulfilled.

3、安装依赖cairocffi:

1)pip install cairocffi安装,会遇到如下问题:

$ pip install cairocffi 
Collecting cairocffi
  Downloading https://files.pythonhosted.org/packages/62/be/ad4d422b6f38d99b09ad6d046ab725e8ccac5fefd9ca256ca35a80dbf3c6/cairocffi-0.9.0.tar.gz (84kB)
    100% |████████████████████████████████| 92kB 85kB/s 
    Complete output from command python setup.py egg_info:
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    c/_cffi_backend.c:2:20: fatal error: Python.h: No such file or directory
     #include <Python.h>
                        ^
    compilation terminated.
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-YUSkTq/cairocffi/setup.py", line 51, in <module>
        'test': ['pytest-runner', 'pytest-cov'],
      File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
        _setup_distribution = dist = klass(attrs)
      File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 265, in __init__
        self.fetch_build_eggs(attrs.pop('setup_requires'))
      File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 289, in fetch_build_eggs
        parse_requirements(requires), installer=self.fetch_build_egg
      File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 618, in resolve
        dist = best[req.key] = env.best_match(req, self, installer)
      File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 862, in best_match
        return self.obtain(req, installer) # try and download/install
      File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 874, in obtain
        return installer(requirement)
      File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 339, in fetch_build_egg
        return cmd.easy_install(req)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 623, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 653, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 849, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1130, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1118, in run_setup
        raise DistutilsError("Setup script exited with %s" % (v.args[0],))
    distutils.errors.DistutilsError: Setup script exited with error: command 'gcc' failed with exit status 1
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-YUSkTq/cairocffi/
You are using pip version 8.1.2, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

解决:根据提示,我们需要升级pip。

$ pip install --upgrade pip
$ pip -V
pip 18.0 from /usr/lib/python2.7/site-packages/pip (python 2.7)

2)继续pip install cairocffi安装,可能会遇到如下问题:

$ pip install cairocffi
Collecting cairocffi
  Using cached https://files.pythonhosted.org/packages/62/be/ad4d422b6f38d99b09ad6d046ab725e8ccac5fefd9ca256ca35a80dbf3c6/cairocffi-0.9.0.tar.gz
    Complete output from command python setup.py egg_info:
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    Package libffi was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libffi.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'libffi' found
    c/_cffi_backend.c:2:20: fatal error: Python.h: No such file or directory
     #include <Python.h>
                        ^
    compilation terminated.
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-RHjhWE/cairocffi/setup.py", line 51, in <module>
        'test': ['pytest-runner', 'pytest-cov'],
      File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
        _setup_distribution = dist = klass(attrs)
      File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 265, in __init__
        self.fetch_build_eggs(attrs.pop('setup_requires'))
      File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 289, in fetch_build_eggs
        parse_requirements(requires), installer=self.fetch_build_egg
      File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 618, in resolve
        dist = best[req.key] = env.best_match(req, self, installer)
      File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 862, in best_match
        return self.obtain(req, installer) # try and download/install
      File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 874, in obtain
        return installer(requirement)
      File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 339, in fetch_build_egg
        return cmd.easy_install(req)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 623, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 653, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 849, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1130, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1118, in run_setup
        raise DistutilsError("Setup script exited with %s" % (v.args[0],))
    distutils.errors.DistutilsError: Setup script exited with error: command 'gcc' failed with exit status 1
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-RHjhWE/cairocffi/

解决:

$ yum install libffi-devel

3)继续安装,可能会遇到如下问题:

$ pip install cairocffi
Collecting cairocffi
  Using cached https://files.pythonhosted.org/packages/62/be/ad4d422b6f38d99b09ad6d046ab725e8ccac5fefd9ca256ca35a80dbf3c6/cairocffi-0.9.0.tar.gz
    Complete output from command python setup.py egg_info:
    c/_cffi_backend.c:2:20: fatal error: Python.h: No such file or directory
     #include <Python.h>
                        ^
    compilation terminated.
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-uMxkrR/cairocffi/setup.py", line 51, in <module>
        'test': ['pytest-runner', 'pytest-cov'],
      File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
        _setup_distribution = dist = klass(attrs)
      File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 265, in __init__
        self.fetch_build_eggs(attrs.pop('setup_requires'))
      File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 289, in fetch_build_eggs
        parse_requirements(requires), installer=self.fetch_build_egg
      File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 618, in resolve
        dist = best[req.key] = env.best_match(req, self, installer)
      File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 862, in best_match
        return self.obtain(req, installer) # try and download/install
      File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 874, in obtain
        return installer(requirement)
      File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 339, in fetch_build_egg
        return cmd.easy_install(req)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 623, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 653, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 849, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1130, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1118, in run_setup
        raise DistutilsError("Setup script exited with %s" % (v.args[0],))
    distutils.errors.DistutilsError: Setup script exited with error: command 'gcc' failed with exit status 1
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-uMxkrR/cairocffi/

解决:

yum install python-devel

至此,一般就可以成功安装cairocffi了。

$ pip install cairocffi
Collecting cairocffi
  Using cached https://files.pythonhosted.org/packages/62/be/ad4d422b6f38d99b09ad6d046ab725e8ccac5fefd9ca256ca35a80dbf3c6/cairocffi-0.9.0.tar.gz
Collecting cffi>=1.1.0 (from cairocffi)
  Downloading https://files.pythonhosted.org/packages/14/dd/3e7a1e1280e7d767bd3fa15791759c91ec19058ebe31217fe66f3e9a8c49/cffi-1.11.5-cp27-cp27mu-manylinux1_x86_64.whl (407kB)
    100% |████████████████████████████████| 409kB 128kB/s 
Collecting pycparser (from cffi>=1.1.0->cairocffi)
  Downloading https://files.pythonhosted.org/packages/8c/2d/aad7f16146f4197a11f8e91fb81df177adcc2073d36a17b1491fd09df6ed/pycparser-2.18.tar.gz (245kB)
    100% |████████████████████████████████| 256kB 137kB/s 
Installing collected packages: pycparser, cffi, cairocffi
  Running setup.py install for pycparser ... done
  Running setup.py install for cairocffi ... done
Successfully installed cairocffi-0.9.0 cffi-1.11.5 pycparser-2.18

4)有的时候,在上面依赖都解决后安装时会遇到网络问题:

$ pip install cairocffi
Collecting cairocffi

  Using cached https://files.pythonhosted.org/packages/62/be/ad4d422b6f38d99b09ad6d046ab725e8ccac5fefd9ca256ca35a80dbf3c6/cairocffi-0.9.0.tar.gz
    Complete output from command python setup.py egg_info:
    
    Installed /tmp/pip-install-TFaywH/cairocffi/cffi-1.11.5-py2.7-linux-x86_64.egg
    Searching for pycparser
    Reading https://pypi.python.org/simple/pycparser/
    Best match: pycparser 2.18
    Downloading https://files.pythonhosted.org/packages/8c/2d/aad7f16146f4197a11f8e91fb81df177adcc2073d36a17b1491fd09df6ed/pycparser-2.18.tar.gz#sha256=99a8ca03e29851d96616ad0404b4aad7d9ee16f25c9f9708a11faf2810f7b226
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-TFaywH/cairocffi/setup.py", line 51, in <module>
        'test': ['pytest-runner', 'pytest-cov'],
      File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
        _setup_distribution = dist = klass(attrs)
      File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 265, in __init__
        self.fetch_build_eggs(attrs.pop('setup_requires'))
      File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 289, in fetch_build_eggs
        parse_requirements(requires), installer=self.fetch_build_egg
      File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 618, in resolve
        dist = best[req.key] = env.best_match(req, self, installer)
      File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 862, in best_match
        return self.obtain(req, installer) # try and download/install
      File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 874, in obtain
        return installer(requirement)
      File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 339, in fetch_build_egg
        return cmd.easy_install(req)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 611, in easy_install
        self.local_index
      File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 598, in fetch_distribution
        return dist.clone(location=self.download(dist.location, tmpdir))
      File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 515, in download
        found = self._download_url(scheme.group(1), spec, tmpdir)
      File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 762, in _download_url
        return self._attempt_download(url, filename)
      File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 768, in _attempt_download
        headers = self._download_to(url, filename)
      File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 662, in _download_to
        fp = self.open_url(url)
      File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 716, in open_url
        % (url, v.reason))
    distutils.errors.DistutilsError: Download error for https://files.pythonhosted.org/packages/8c/2d/aad7f16146f4197a11f8e91fb81df177adcc2073d36a17b1491fd09df6ed/pycparser-2.18.tar.gz#sha256=99a8ca03e29851d96616ad0404b4aad7d9ee16f25c9f9708a11faf2810f7b226: [Errno 101] Network is unreachable
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-TFaywH/cairocffi/

解决:

根据报错中的连接地址手工下载pycparser-2.18.tar.gz,然后解压python setup.py install 执行后即可;还有可能cffi-1.11.5.tar.gz无法下载,同样根据报错连接地址手工下载,然后解压python setup.py install 执行后即可。

5)安装完pip install cairocffi后,再次执行$ python check-dependencies.py 检查依赖:

$ python check-dependencies.py
Traceback (most recent call last):
  File "check-dependencies.py", line 31, in <module>
    import cairocffi as cairo
  File "/usr/lib/python2.7/site-packages/cairocffi/__init__.py", line 41, in <module>
    cairo = dlopen(ffi, 'cairo', 'cairo-2', 'cairo-gobject-2')
  File "/usr/lib/python2.7/site-packages/cairocffi/__init__.py", line 38, in dlopen
    raise OSError("dlopen() failed to load a library: %s" % ' / '.join(names))
OSError: dlopen() failed to load a library: cairo / cairo-2 / cairo-gobject-2

我们发现报错了,在安装cairocffi之前还没有报错。解决如下:

yum install cairo-devel

再次执行

$ python check-dependencies.py 
[REQUIRED] Unable to import the 'django' module, do you have Django installed for python 2?
[REQUIRED] Unable to import the 'pytz' module, do you have pytz module installed for python 2?
[REQUIRED] Unable to import the 'pyparsing' module, do you have pyparsing module installed for python 2?
[REQUIRED] Unable to import the 'tagging' module, do you have django-tagging installed for python 2?
[OPTIONAL] Unable to import the 'memcache' module, do you have python-memcached installed for python 2? This feature is not required but greatly improves performance.
[OPTIONAL] Unable to import the 'ldap' module, do you have python-ldap installed for python 2? Without python-ldap, you will not be able to use LDAP authentication in the graphite webapp.
[OPTIONAL] Unable to import the 'txamqp' module, this is required if you want to use AMQP as an input to Carbon. Note that txamqp requires python 2.5 or greater.
[OPTIONAL] Unable to import the 'python-rrdtool' module, this is required for reading RRD.
[OPTIONAL] Unable to import the 'whitenoise' module. This is useful for serving static files.
[OPTIONAL] Unable to import the 'pyhash' module. This is useful for fnv1_ch hashing support.
6 optional dependencies not met. Please consider the optional items before proceeding.
4 necessary dependencies not met. Graphite will not function until these dependencies are fulfilled.

4、安装其他依赖:

1)django:

查看官网 https://graphite.readthedocs.io/en/latest/install.html#dependencies ,至少需要django1.8

pip install Django==1.8

2)安装pytz:

pip install pytz
Collecting pytz
  Downloading https://files.pythonhosted.org/packages/30/4e/27c34b62430286c6d59177a0842ed90dc789ce5d1ed740887653b898779a/pytz-2018.5-py2.py3-none-any.whl (510kB)
    100% |████████████████████████████████| 512kB 242kB/s 
Installing collected packages: pytz
Successfully installed pytz-2018.5

3)安装pyparsing:

$ pip install pyparsing
Collecting pyparsing
  Downloading https://files.pythonhosted.org/packages/6a/8a/718fd7d3458f9fab8e67186b00abdd345b639976bc7fb3ae722e1b026a50/pyparsing-2.2.0-py2.py3-none-any.whl (56kB)
    100% |████████████████████████████████| 61kB 131kB/s 
Installing collected packages: pyparsing
Successfully installed pyparsing-2.2.0

4)安装jdango-tagging:

$ pip install django-tagging
Collecting django-tagging
  Downloading https://files.pythonhosted.org/packages/5e/fc/9d095602bf5d2edcbc2c5721e3d243028544575a145d84ca1ec50f7e2fc1/django_tagging-0.4.6-py2.py3-none-any.whl
Installing collected packages: django-tagging
Successfully installed django-tagging-0.4.6

然后再次检查依赖:

$ python check-dependencies.py 
[OPTIONAL] Unable to import the 'memcache' module, do you have python-memcached installed for python 2? This feature is not required but greatly improves performance.
[OPTIONAL] Unable to import the 'ldap' module, do you have python-ldap installed for python 2? Without python-ldap, you will not be able to use LDAP authentication in the graphite webapp.
[OPTIONAL] Unable to import the 'txamqp' module, this is required if you want to use AMQP as an input to Carbon. Note that txamqp requires python 2.5 or greater.
[OPTIONAL] Unable to import the 'python-rrdtool' module, this is required for reading RRD.
[OPTIONAL] Unable to import the 'whitenoise' module. This is useful for serving static files.
[OPTIONAL] Unable to import the 'pyhash' module. This is useful for fnv1_ch hashing support.
6 optional dependencies not met. Please consider the optional items before proceeding.
3 necessary dependencies not met. Graphite will not function until these dependencies are fulfilled.

三、配置:


1、重命名配置文件:

进入到/opt/graphite/conf目录下,里面有很多例子,我们需要把后缀example去掉

cd /opt/graphite/conf
cp aggregation-rules.conf.example aggregation-rules.conf
cp blacklist.conf.example blacklist.conf
cp carbon.conf.example carbon.conf
cp carbon.amqp.conf.example carbon.amqp.conf
cp relay-rules.conf.example relay-rules.conf
cp rewrite-rules.conf.example rewrite-rules.conf
cp storage-schemas.conf.example storage-schemas.conf
cp storage-aggregation.conf.example storage-aggregation.conf
cp whitelist.conf.example whitelist.conf


2、启动carbon-cache:

#cd /opt/graphite/bin
./carbon-cache.py start

1)启动报错:

$ ./carbon-cache.py start
Traceback (most recent call last):
  File "./carbon-cache.py", line 28, in <module>
    from carbon.util import run_twistd_plugin  # noqa
  File "/opt/graphite/lib/carbon/util.py", line 15, in <module>
    from twisted.python.util import initgroups
ImportError: No module named twisted.python.util

解决:安装twisted

$ pip install twisted
Collecting twisted
  Downloading https://files.pythonhosted.org/packages/90/50/4c315ce5d119f67189d1819629cae7908ca0b0a6c572980df5cc6942bc22/Twisted-18.7.0.tar.bz2 (3.1MB)
    100% |████████████████████████████████| 3.1MB 133kB/s 
Collecting zope.interface>=4.4.2 (from twisted)
  Downloading https://files.pythonhosted.org/packages/ac/8a/657532df378c2cd2a1fe6b12be3b4097521570769d4852ec02c24bd3594e/zope.interface-4.5.0.tar.gz (151kB)
    100% |████████████████████████████████| 153kB 89kB/s 
Collecting constantly>=15.1 (from twisted)
  Downloading https://files.pythonhosted.org/packages/b9/65/48c1909d0c0aeae6c10213340ce682db01b48ea900a7d9fce7a7910ff318/constantly-15.1.0-py2.py3-none-any.whl
Collecting incremental>=16.10.1 (from twisted)
  Downloading https://files.pythonhosted.org/packages/f5/1d/c98a587dc06e107115cf4a58b49de20b19222c83d75335a192052af4c4b7/incremental-17.5.0-py2.py3-none-any.whl
Collecting Automat>=0.3.0 (from twisted)
  Downloading https://files.pythonhosted.org/packages/a3/86/14c16bb98a5a3542ed8fed5d74fb064a902de3bdd98d6584b34553353c45/Automat-0.7.0-py2.py3-none-any.whl
Collecting hyperlink>=17.1.1 (from twisted)
  Downloading https://files.pythonhosted.org/packages/a7/b6/84d0c863ff81e8e7de87cff3bd8fd8f1054c227ce09af1b679a8b17a9274/hyperlink-18.0.0-py2.py3-none-any.whl
Collecting PyHamcrest>=1.9.0 (from twisted)
  Downloading https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl (52kB)
    100% |████████████████████████████████| 61kB 61kB/s 
Collecting attrs>=17.4.0 (from twisted)
  Downloading https://files.pythonhosted.org/packages/3a/e1/5f9023cc983f1a628a8c2fd051ad19e76ff7b142a0faf329336f9a62a514/attrs-18.2.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools in /usr/lib/python2.7/site-packages (from zope.interface>=4.4.2->twisted) (0.9.8)
Collecting six (from Automat>=0.3.0->twisted)
  Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting idna>=2.5 (from hyperlink>=17.1.1->twisted)
  Downloading https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl (58kB)
    100% |████████████████████████████████| 61kB 19kB/s 
Installing collected packages: zope.interface, constantly, incremental, attrs, six, Automat, idna, hyperlink, PyHamcrest, twisted
  Running setup.py install for zope.interface ... done
  Running setup.py install for twisted ... done
Successfully installed Automat-0.7.0 PyHamcrest-1.9.0 attrs-18.2.0 constantly-15.1.0 hyperlink-18.0.0 idna-2.7 incremental-17.5.0 six-1.11.0 twisted-18.7.0 zope.interface-4.5.0

2)在pip install twisted的时候,有可能网络问题无法下载依赖:

Collecting twisted
  Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x3357a50>: Failed to establish a new connection: [Errno 110] Connection timed out',)': /packages/90/50/4c315ce5d119f67189d1819629cae7908ca0b0a6c572980df5cc6942bc22/Twisted-18.7.0.tar.bz2
  Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x3357d50>: Failed to establish a new connection: [Errno 110] Connection timed out',)': /packages/90/50/4c315ce5d119f67189d1819629cae7908ca0b0a6c572980df5cc6942bc22/Twisted-18.7.0.tar.bz2
  Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x3357f50>: Failed to establish a new connection: [Errno 110] Connection timed out',)': /packages/90/50/4c315ce5d119f67189d1819629cae7908ca0b0a6c572980df5cc6942bc22/Twisted-18.7.0.tar.bz2
  Downloading https://files.pythonhosted.org/packages/90/50/4c315ce5d119f67189d1819629cae7908ca0b0a6c572980df5cc6942bc22/Twisted-18.7.0.tar.bz2 (3.1MB)
    100% |████████████████████████████████| 3.1MB 5.2MB/s 
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-QrXqMw/twisted/setup.py", line 20, in <module>
        setuptools.setup(**_setup["getSetupArgs"]())
      File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
        _setup_distribution = dist = klass(attrs)
      File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 265, in __init__
        self.fetch_build_eggs(attrs.pop('setup_requires'))
      File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 289, in fetch_build_eggs
        parse_requirements(requires), installer=self.fetch_build_egg
      File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 618, in resolve
        dist = best[req.key] = env.best_match(req, self, installer)
      File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 862, in best_match
        return self.obtain(req, installer) # try and download/install
      File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 874, in obtain
        return installer(requirement)
      File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 339, in fetch_build_egg
        return cmd.easy_install(req)
      File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 611, in easy_install
        self.local_index
      File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 598, in fetch_distribution
        return dist.clone(location=self.download(dist.location, tmpdir))
      File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 515, in download
        found = self._download_url(scheme.group(1), spec, tmpdir)
      File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 762, in _download_url
        return self._attempt_download(url, filename)
      File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 768, in _attempt_download
        headers = self._download_to(url, filename)
      File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 662, in _download_to
        fp = self.open_url(url)
      File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 716, in open_url
        % (url, v.reason))
    distutils.errors.DistutilsError: Download error for https://files.pythonhosted.org/packages/8f/26/02c4016aa95f45479eea37c90c34f8fab6775732ae62587a874b619ca097/incremental-17.5.0.tar.gz#sha256=7b751696aaf36eebfab537e458929e194460051ccad279c72b755a167eebd4b3: [Errno 101] Network is unreachable
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-QrXqMw/twisted/

解决:根据报错提示连接地址,手动下载incremental-17.5.0.tar.gz,然后python setup.py install

至此,可以成功启动carbon。

$ ./carbon-cache.py start
Starting carbon-cache (instance a)

检查进程:

#netstat -nap | grep 2003

发送数据:

# yum install nc
echo "carbon.agents.graphite-tutorial.metricsReceived 28198 `date +%s`" | nc localhost 2003


3、同步数据:

1)第一种方式:

cd /opt/graphite-web/webapp/
cp manage.py /opt/graphite/webapp/
cd /opt/graphite/webapp/
$ python manage.py syncdb

在执行过程中可以设置用户名密码(用于在web界面上登录)




:通过源码安装graphite,manage.py没有被放到/opt/graphite/下,需要手动拷贝过去。

2)第二种方式:

PYTHONPATH=/opt/graphite/webapp django-admin.py migrate --settings=graphite.settings --run-syncdb
Could not import graphite.local_settings, using defaults!
/opt/graphite/webapp/graphite/settings.py:332: UserWarning: SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security
  warn('SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security')
Operations to perform:
  Synchronize unmigrated apps: functions, render, staticfiles, whitelist, metrics, composer, browser
  Apply all migrations: account, sessions, admin, tags, auth, url_shortener, contenttypes, dashboard, events, tagging
Synchronizing apps without migrations:
  Creating tables...
    Running deferred SQL...
  Installing custom SQL...
Running migrations:
  Rendering model states... DONE
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying account.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying dashboard.0001_initial... OK
  Applying events.0001_initial... OK
  Applying sessions.0001_initial... OK
  Applying tagging.0001_initial... OK
  Applying tagging.0002_on_delete... OK
  Applying tagging.0003_adapt_max_tag_length... OK
  Applying tags.0001_initial... OK
  Applying url_shortener.0001_initial... OK


注:执行django-admin.py syncdb –settings=graphite.settings提示找不到syncdb,原因是django 1.7以后需要使用 migrate

四、启动graphite:

graphite指的是一个大的技术集合,她依赖了carbon、whisper、graphite-web三个组件,上面我们已经把carbon启动了。这一步启动graphite其实指的是启动graphite-web,graphite-web是一个python写的web项目,使用了djiango框架。此外,graphite-web除了提供界面外,还提供了很多restful的api接口(比如查询数据、聚合数据等)

1、最简单的方式启动:

# cd /opt/graphite
# PYTHONPATH=`pwd`/storage/whisper ./bin/run-graphite-devel-server.py --port=8085 --libs=`pwd`/webapp /opt/graphite 1>/opt/graphite/storage/log/webapp/process.log 2>&1 &
# tail -f /opt/graphite/storage/log/webapp/process.log

1)可能会遇到如下错误:

Traceback (most recent call last):
  File "/usr/lib64/python2.7/wsgiref/handlers.py", line 85, in run
    self.result = application(self.environ, self.start_response)
  File "/usr/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 189, in __call__
    response = self.get_response(request)
  File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 218, in get_response
    response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
  File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 264, in handle_uncaught_exception
    if resolver.urlconf_module is None:
  File "/usr/lib/python2.7/site-packages/django/core/urlresolvers.py", line 396, in urlconf_module
    self._urlconf_module = import_module(self.urlconf_name)
  File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/opt/graphite/webapp/graphite/urls.py", line 22, in <module>
    url('^render', include('graphite.render.urls')),
  File "/usr/lib/python2.7/site-packages/django/conf/urls/__init__.py", line 33, in include
    urlconf_module = import_module(urlconf_module)
  File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/opt/graphite/webapp/graphite/render/urls.py", line 16, in <module>
    from . import views
  File "/opt/graphite/webapp/graphite/render/views.py", line 28, in <module>
    from graphite.storage import extractForwardHeaders
  File "/opt/graphite/webapp/graphite/storage.py", line 27, in <module>
    from graphite.readers import MultiReader
  File "/opt/graphite/webapp/graphite/readers/__init__.py", line 4, in <module>
    from graphite.readers.whisper import WhisperReader, GzippedWhisperReader  # noqa # pylint: disable=unused-import
  File "/opt/graphite/webapp/graphite/readers/whisper.py", line 9, in <module>
    from scandir import scandir, stat  # noqa # pylint: disable=unused-import
ImportError: No module named scandir

解决:

pip install scandir

2)再次启动,可能会遇到如下错误:

ImportError: No module named urllib3
解决:
pip install urllib3

3)解决上面依赖后,即可成功启动graphite-web:

我们看启动命令是通过默认的run-graphite-devel-server.py启动python web程序,同时指定了8085端口。在高版本的graphite-web中,通过这种方式启动,在浏览器中输入http:ip:8085,无法访问静态图片、页面等。查看日志,发现404

[17/Sep/2018 03:23:02]"GET / HTTP/1.1" 200 598
[17/Sep/2018 03:23:02]"GET /browser/header HTTP/1.1" 200 1166
[17/Sep/2018 03:23:02]"GET /composer? HTTP/1.1" 200 1643
[17/Sep/2018 03:23:03]"GET /static/img/carbon-fiber.png HTTP/1.1" 404 101
[17/Sep/2018 03:23:03]"GET /static/img/graphite-logo.png HTTP/1.1" 404 102
[17/Sep/2018 03:23:03]"GET /static/js/ext/resources/css/ext-all.css HTTP/1.1" 404 113
[17/Sep/2018 03:23:03]"GET /static/js/browser.js HTTP/1.1" 404 94
[17/Sep/2018 03:23:03]"GET /static/js/composer_widgets.js HTTP/1.1" 404 103
[17/Sep/2018 03:23:03]"GET /static/js/ext/ext-all.js HTTP/1.1" 404 98
[17/Sep/2018 03:23:03]"GET /static/js/ext/adapter/ext/ext-base.js HTTP/1.1" 404 111
[17/Sep/2018 03:23:03]"GET /static/js/composer.js HTTP/1.1" 404 95
[17/Sep/2018 03:23:03]"GET /static/js/completer.js HTTP/1.1" 404 96
[17/Sep/2018 03:23:03]"GET /static/js/ext/ext-all.js HTTP/1.1" 404 98
[17/Sep/2018 03:23:03]"GET /static/js/browser.js HTTP/1.1" 404 94
[17/Sep/2018 03:23:03]"GET /static/js/composer_widgets.js HTTP/1.1" 404 103
[17/Sep/2018 03:23:03]"GET /static/js/composer.js HTTP/1.1" 404 95
[17/Sep/2018 03:23:03]"GET /static/js/completer.js HTTP/1.1" 404 96

我们通过下面命令来收集静态信息:

$  PYTHONPATH=/opt/graphite/webapp django-admin.py collectstatic –noinput –settings=graphite.settings

$  PYTHONPATH=/opt/graphite/webapp django-admin.py collectstatic --noinput --settings=graphite.settings
/opt/graphite/webapp/graphite/settings.py:332: UserWarning: SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security
  warn('SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security')
Copying '/opt/graphite/webapp/content/js/browser.js'
Copying '/opt/graphite/webapp/content/js/completer.js'
Copying '/opt/graphite/webapp/content/js/composer.js'
Copying '/opt/graphite/webapp/content/js/composer_widgets.js'
Copying '/opt/graphite/webapp/content/js/dashboard.js'
Copying '/opt/graphite/webapp/content/js/ext/ext-all-debug.js'
Copying '/opt/graphite/webapp/content/js/ext/ext-all.js'
Copying '/opt/graphite/webapp/content/js/ext/resources/images/default/gradient-bg.gif'
....

执行该命令后会在/opt/graphite下生成static目录,里面包含了js、css、html等内容,如下:

$ pwd
/opt/graphite/static
$ ll
drwxr-xr-x 5 root root   35 Sep 17 11:22 admin
drwxr-xr-x 4 root root  156 Sep 17 11:22 css
drwxr-xr-x 2 root root   53 Sep 17 11:22 html
drwxr-xr-x 2 root root 4096 Sep 17 11:22 img
drwxr-xr-x 4 root root  125 Sep 17 11:22 js

执行了这个让然无法访问静态文件,我们通过uwsgi方式来解决。

2、启动方式二:

除了上面的方式启动graphite-web,还可以用下面的方式:

PYTHONPATH=/opt/graphite/webapp/ django-admin.py runserver 10.39.12.24:8085 --settings=graphite.settings

这里使用了djanggo-admin,本质和上面的一样。同样,也是无法访问到静态资源。


3、使用uwsgi+nginx方式启动

上面两种方式都是测试环境上来启动的,生产环境大都是Nginx作为web服务器(nginx来部署graphite),这里需要安装下载uwsgi。本人对python不是很熟悉,大致上理解为:上面两种像是通过内置jetty来启动java-web项目;这种方式是通过tomcat部署java-web项目,我们可以通过设置tomcat的参数来优化java-web。

1)安装nginx:

2)安装uwsgi:

$ pip install uwsgi
Collecting uwsgi
  Downloading https://files.pythonhosted.org/packages/a2/c9/a2d5737f63cd9df4317a4acc15d1ddf4952e28398601d8d7d706c16381e0/uwsgi-2.0.17.1.tar.gz (800kB)
    100% |████████████████████████████████| 808kB 739kB/s 
Installing collected packages: uwsgi
  Running setup.py install for uwsgi ... done
Successfully installed uwsgi-2.0.17.1

3)配置uwsgi文件:

vim /opt/graphite/webapp/graphite.ini(这个文件不存在)

[uwsgi]
post-buffering = 65536
buffer-size = 65536
chdir=/opt/graphite/webapp/
module=graphite.wsgi
master=true
pidfile=/var/run/graphite.pid
vacuum=true
max-requests=20000
daemonize=/opt/graphite/storage/log/webapp/access.log
processes=32
harakiri=10
harakiri-verbose=true
socket=127.0.0.1:9090
stats=127.0.0.1:1716
listen=1024


注意

:在graphite运行一段时间之后可能会出现大量超时的情况,此时需要适当的增加processes数量,但是有一点极其关键需要增加process回收参数设置:harakiri-verbose=true表示超时回收机制,harakiri=10表示超时10即回收

4)配置nginx:

location / {
		#root   html;
		#index  index.html index.htm;
		uwsgi_pass 127.0.0.1:9090;
		include uwsgi_params;
	}

location /static/ {
	alias /opt/graphite/webapp/content/;
	index index.html index.html;
}

5)启动nginx,然后通过http://ip 即可访问到graphite-web界面了。

6)grafana连接graphite:

  • 如果通过前两种方式启动的graphite-web,那么在grafana的datasource中设置http://ip:8085;
  • 如果通过nginx+uwsgi启动的graphite-web,那么在grafana的DataSource中设置http://ip/ 即可。

注:grafana本质上是通过graphite-web提供的restful-api来获取graphite中carbon存储的数据,然后再展示到界面上。

参考:


https://graphite.readthedocs.io/en/latest/install-source.html


https://blog.csdn.net/weixin_39922154/article/details/79132147


http://www.infoq.com/cn/articles/graphite-intro


https://my.oschina.net/u/1263964/blog/711998


https://uwsgi-docs-zh.readthedocs.io/zh_CN/latest/tutorials/Django_and_nginx.html



版权声明:本文为liuxiao723846原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。