一、python部分:
0、系统:
$ cat /etc/redhat-release
CentOS Linux release 7.2.1511 (Core)
1、检查系统python是否2.7以上(最新graphite需要python至少2.7):
$ python -V
Python 2.7.5
2、安装pip:
yum install python-pip
$ pip -V
pip 8.1.2 from /usr/lib/python2.7/site-packages (python 2.7)
这里安装后的版本是8,后面还需要再升级一下pip,因为安装cairocffi时需要。
二、下载相关源码、并且安装:
1、git下载graphite相关源码下载、安装:
cd /opt
下载源码:
git clone https://github.com/graphite-project/graphite-web.git
git clone https://github.com/graphite-project/carbon.git
git clone https://github.com/graphite-project/whisper.git
然后通过python setup.py install安装:
cd /opt/graphite-web
python setup.py install
cd carbon
python setup.py install
cd whisper
python setup.py install
这时在/opt下多了一个graphite目录,/opt目录如下
drwxr-xr-x 9 root root 4096 Sep 17 10:30 carbon
drwxr-xr-x 9 root root 92 Sep 17 11:22 graphite
drwxr-xr-x 11 root root 4096 Sep 17 10:29 graphite-web
drwxr-xr-x 3 root root 19 Sep 17 09:32 soft
drwxr-xr-x 6 root root 4096 Sep 17 10:30 whisper
2、检查依赖:
cd /opt/graphite-web
$ python check-dependencies.py
[REQUIRED] Unable to import the 'cairocffi' module, attempting to fall back to pycairo
[REQUIRED] Unable to import the 'cairo' module, do you have pycairo installed for python 2?
[REQUIRED] Unable to import the 'django' module, do you have Django installed for python 2?
[REQUIRED] Unable to import the 'pytz' module, do you have pytz module installed for python 2?
[REQUIRED] Unable to import the 'pyparsing' module, do you have pyparsing module installed for python 2?
[REQUIRED] Unable to import the 'tagging' module, do you have django-tagging installed for python 2?
[OPTIONAL] Unable to import the 'memcache' module, do you have python-memcached installed for python 2? This feature is not required but greatly improves performance.
[OPTIONAL] Unable to import the 'ldap' module, do you have python-ldap installed for python 2? Without python-ldap, you will not be able to use LDAP authentication in the graphite webapp.
[OPTIONAL] Unable to import the 'txamqp' module, this is required if you want to use AMQP as an input to Carbon. Note that txamqp requires python 2.5 or greater.
[OPTIONAL] Unable to import the 'python-rrdtool' module, this is required for reading RRD.
[OPTIONAL] Unable to import the 'whitenoise' module. This is useful for serving static files.
[OPTIONAL] Unable to import the 'pyhash' module. This is useful for fnv1_ch hashing support.
6 optional dependencies not met. Please consider the optional items before proceeding.
5 necessary dependencies not met. Graphite will not function until these dependencies are fulfilled.
3、安装依赖cairocffi:
1)pip install cairocffi安装,会遇到如下问题:
$ pip install cairocffi
Collecting cairocffi
Downloading https://files.pythonhosted.org/packages/62/be/ad4d422b6f38d99b09ad6d046ab725e8ccac5fefd9ca256ca35a80dbf3c6/cairocffi-0.9.0.tar.gz (84kB)
100% |████████████████████████████████| 92kB 85kB/s
Complete output from command python setup.py egg_info:
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
c/_cffi_backend.c:2:20: fatal error: Python.h: No such file or directory
#include <Python.h>
^
compilation terminated.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-YUSkTq/cairocffi/setup.py", line 51, in <module>
'test': ['pytest-runner', 'pytest-cov'],
File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 265, in __init__
self.fetch_build_eggs(attrs.pop('setup_requires'))
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 289, in fetch_build_eggs
parse_requirements(requires), installer=self.fetch_build_egg
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 618, in resolve
dist = best[req.key] = env.best_match(req, self, installer)
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 862, in best_match
return self.obtain(req, installer) # try and download/install
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 874, in obtain
return installer(requirement)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 339, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 623, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 653, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 849, in install_eggs
return self.build_and_install(setup_script, setup_base)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1130, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1118, in run_setup
raise DistutilsError("Setup script exited with %s" % (v.args[0],))
distutils.errors.DistutilsError: Setup script exited with error: command 'gcc' failed with exit status 1
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-YUSkTq/cairocffi/
You are using pip version 8.1.2, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
解决:根据提示,我们需要升级pip。
$ pip install --upgrade pip
$ pip -V
pip 18.0 from /usr/lib/python2.7/site-packages/pip (python 2.7)
2)继续pip install cairocffi安装,可能会遇到如下问题:
$ pip install cairocffi
Collecting cairocffi
Using cached https://files.pythonhosted.org/packages/62/be/ad4d422b6f38d99b09ad6d046ab725e8ccac5fefd9ca256ca35a80dbf3c6/cairocffi-0.9.0.tar.gz
Complete output from command python setup.py egg_info:
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing `libffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
c/_cffi_backend.c:2:20: fatal error: Python.h: No such file or directory
#include <Python.h>
^
compilation terminated.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-RHjhWE/cairocffi/setup.py", line 51, in <module>
'test': ['pytest-runner', 'pytest-cov'],
File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 265, in __init__
self.fetch_build_eggs(attrs.pop('setup_requires'))
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 289, in fetch_build_eggs
parse_requirements(requires), installer=self.fetch_build_egg
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 618, in resolve
dist = best[req.key] = env.best_match(req, self, installer)
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 862, in best_match
return self.obtain(req, installer) # try and download/install
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 874, in obtain
return installer(requirement)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 339, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 623, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 653, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 849, in install_eggs
return self.build_and_install(setup_script, setup_base)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1130, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1118, in run_setup
raise DistutilsError("Setup script exited with %s" % (v.args[0],))
distutils.errors.DistutilsError: Setup script exited with error: command 'gcc' failed with exit status 1
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-RHjhWE/cairocffi/
解决:
$ yum install libffi-devel
3)继续安装,可能会遇到如下问题:
$ pip install cairocffi
Collecting cairocffi
Using cached https://files.pythonhosted.org/packages/62/be/ad4d422b6f38d99b09ad6d046ab725e8ccac5fefd9ca256ca35a80dbf3c6/cairocffi-0.9.0.tar.gz
Complete output from command python setup.py egg_info:
c/_cffi_backend.c:2:20: fatal error: Python.h: No such file or directory
#include <Python.h>
^
compilation terminated.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-uMxkrR/cairocffi/setup.py", line 51, in <module>
'test': ['pytest-runner', 'pytest-cov'],
File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 265, in __init__
self.fetch_build_eggs(attrs.pop('setup_requires'))
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 289, in fetch_build_eggs
parse_requirements(requires), installer=self.fetch_build_egg
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 618, in resolve
dist = best[req.key] = env.best_match(req, self, installer)
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 862, in best_match
return self.obtain(req, installer) # try and download/install
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 874, in obtain
return installer(requirement)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 339, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 623, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 653, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 849, in install_eggs
return self.build_and_install(setup_script, setup_base)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1130, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1118, in run_setup
raise DistutilsError("Setup script exited with %s" % (v.args[0],))
distutils.errors.DistutilsError: Setup script exited with error: command 'gcc' failed with exit status 1
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-uMxkrR/cairocffi/
解决:
yum install python-devel
至此,一般就可以成功安装cairocffi了。
$ pip install cairocffi
Collecting cairocffi
Using cached https://files.pythonhosted.org/packages/62/be/ad4d422b6f38d99b09ad6d046ab725e8ccac5fefd9ca256ca35a80dbf3c6/cairocffi-0.9.0.tar.gz
Collecting cffi>=1.1.0 (from cairocffi)
Downloading https://files.pythonhosted.org/packages/14/dd/3e7a1e1280e7d767bd3fa15791759c91ec19058ebe31217fe66f3e9a8c49/cffi-1.11.5-cp27-cp27mu-manylinux1_x86_64.whl (407kB)
100% |████████████████████████████████| 409kB 128kB/s
Collecting pycparser (from cffi>=1.1.0->cairocffi)
Downloading https://files.pythonhosted.org/packages/8c/2d/aad7f16146f4197a11f8e91fb81df177adcc2073d36a17b1491fd09df6ed/pycparser-2.18.tar.gz (245kB)
100% |████████████████████████████████| 256kB 137kB/s
Installing collected packages: pycparser, cffi, cairocffi
Running setup.py install for pycparser ... done
Running setup.py install for cairocffi ... done
Successfully installed cairocffi-0.9.0 cffi-1.11.5 pycparser-2.18
4)有的时候,在上面依赖都解决后安装时会遇到网络问题:
$ pip install cairocffi
Collecting cairocffi
Using cached https://files.pythonhosted.org/packages/62/be/ad4d422b6f38d99b09ad6d046ab725e8ccac5fefd9ca256ca35a80dbf3c6/cairocffi-0.9.0.tar.gz
Complete output from command python setup.py egg_info:
Installed /tmp/pip-install-TFaywH/cairocffi/cffi-1.11.5-py2.7-linux-x86_64.egg
Searching for pycparser
Reading https://pypi.python.org/simple/pycparser/
Best match: pycparser 2.18
Downloading https://files.pythonhosted.org/packages/8c/2d/aad7f16146f4197a11f8e91fb81df177adcc2073d36a17b1491fd09df6ed/pycparser-2.18.tar.gz#sha256=99a8ca03e29851d96616ad0404b4aad7d9ee16f25c9f9708a11faf2810f7b226
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-TFaywH/cairocffi/setup.py", line 51, in <module>
'test': ['pytest-runner', 'pytest-cov'],
File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 265, in __init__
self.fetch_build_eggs(attrs.pop('setup_requires'))
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 289, in fetch_build_eggs
parse_requirements(requires), installer=self.fetch_build_egg
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 618, in resolve
dist = best[req.key] = env.best_match(req, self, installer)
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 862, in best_match
return self.obtain(req, installer) # try and download/install
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 874, in obtain
return installer(requirement)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 339, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 611, in easy_install
self.local_index
File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 598, in fetch_distribution
return dist.clone(location=self.download(dist.location, tmpdir))
File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 515, in download
found = self._download_url(scheme.group(1), spec, tmpdir)
File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 762, in _download_url
return self._attempt_download(url, filename)
File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 768, in _attempt_download
headers = self._download_to(url, filename)
File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 662, in _download_to
fp = self.open_url(url)
File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 716, in open_url
% (url, v.reason))
distutils.errors.DistutilsError: Download error for https://files.pythonhosted.org/packages/8c/2d/aad7f16146f4197a11f8e91fb81df177adcc2073d36a17b1491fd09df6ed/pycparser-2.18.tar.gz#sha256=99a8ca03e29851d96616ad0404b4aad7d9ee16f25c9f9708a11faf2810f7b226: [Errno 101] Network is unreachable
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-TFaywH/cairocffi/
解决:
根据报错中的连接地址手工下载pycparser-2.18.tar.gz,然后解压python setup.py install 执行后即可;还有可能cffi-1.11.5.tar.gz无法下载,同样根据报错连接地址手工下载,然后解压python setup.py install 执行后即可。
5)安装完pip install cairocffi后,再次执行$ python check-dependencies.py 检查依赖:
$ python check-dependencies.py
Traceback (most recent call last):
File "check-dependencies.py", line 31, in <module>
import cairocffi as cairo
File "/usr/lib/python2.7/site-packages/cairocffi/__init__.py", line 41, in <module>
cairo = dlopen(ffi, 'cairo', 'cairo-2', 'cairo-gobject-2')
File "/usr/lib/python2.7/site-packages/cairocffi/__init__.py", line 38, in dlopen
raise OSError("dlopen() failed to load a library: %s" % ' / '.join(names))
OSError: dlopen() failed to load a library: cairo / cairo-2 / cairo-gobject-2
我们发现报错了,在安装cairocffi之前还没有报错。解决如下:
yum install cairo-devel
再次执行
$ python check-dependencies.py
[REQUIRED] Unable to import the 'django' module, do you have Django installed for python 2?
[REQUIRED] Unable to import the 'pytz' module, do you have pytz module installed for python 2?
[REQUIRED] Unable to import the 'pyparsing' module, do you have pyparsing module installed for python 2?
[REQUIRED] Unable to import the 'tagging' module, do you have django-tagging installed for python 2?
[OPTIONAL] Unable to import the 'memcache' module, do you have python-memcached installed for python 2? This feature is not required but greatly improves performance.
[OPTIONAL] Unable to import the 'ldap' module, do you have python-ldap installed for python 2? Without python-ldap, you will not be able to use LDAP authentication in the graphite webapp.
[OPTIONAL] Unable to import the 'txamqp' module, this is required if you want to use AMQP as an input to Carbon. Note that txamqp requires python 2.5 or greater.
[OPTIONAL] Unable to import the 'python-rrdtool' module, this is required for reading RRD.
[OPTIONAL] Unable to import the 'whitenoise' module. This is useful for serving static files.
[OPTIONAL] Unable to import the 'pyhash' module. This is useful for fnv1_ch hashing support.
6 optional dependencies not met. Please consider the optional items before proceeding.
4 necessary dependencies not met. Graphite will not function until these dependencies are fulfilled.
4、安装其他依赖:
1)django:
查看官网 https://graphite.readthedocs.io/en/latest/install.html#dependencies ,至少需要django1.8
pip install Django==1.8
2)安装pytz:
pip install pytz
Collecting pytz
Downloading https://files.pythonhosted.org/packages/30/4e/27c34b62430286c6d59177a0842ed90dc789ce5d1ed740887653b898779a/pytz-2018.5-py2.py3-none-any.whl (510kB)
100% |████████████████████████████████| 512kB 242kB/s
Installing collected packages: pytz
Successfully installed pytz-2018.5
3)安装pyparsing:
$ pip install pyparsing
Collecting pyparsing
Downloading https://files.pythonhosted.org/packages/6a/8a/718fd7d3458f9fab8e67186b00abdd345b639976bc7fb3ae722e1b026a50/pyparsing-2.2.0-py2.py3-none-any.whl (56kB)
100% |████████████████████████████████| 61kB 131kB/s
Installing collected packages: pyparsing
Successfully installed pyparsing-2.2.0
4)安装jdango-tagging:
$ pip install django-tagging
Collecting django-tagging
Downloading https://files.pythonhosted.org/packages/5e/fc/9d095602bf5d2edcbc2c5721e3d243028544575a145d84ca1ec50f7e2fc1/django_tagging-0.4.6-py2.py3-none-any.whl
Installing collected packages: django-tagging
Successfully installed django-tagging-0.4.6
然后再次检查依赖:
$ python check-dependencies.py
[OPTIONAL] Unable to import the 'memcache' module, do you have python-memcached installed for python 2? This feature is not required but greatly improves performance.
[OPTIONAL] Unable to import the 'ldap' module, do you have python-ldap installed for python 2? Without python-ldap, you will not be able to use LDAP authentication in the graphite webapp.
[OPTIONAL] Unable to import the 'txamqp' module, this is required if you want to use AMQP as an input to Carbon. Note that txamqp requires python 2.5 or greater.
[OPTIONAL] Unable to import the 'python-rrdtool' module, this is required for reading RRD.
[OPTIONAL] Unable to import the 'whitenoise' module. This is useful for serving static files.
[OPTIONAL] Unable to import the 'pyhash' module. This is useful for fnv1_ch hashing support.
6 optional dependencies not met. Please consider the optional items before proceeding.
3 necessary dependencies not met. Graphite will not function until these dependencies are fulfilled.
三、配置:
1、重命名配置文件:
进入到/opt/graphite/conf目录下,里面有很多例子,我们需要把后缀example去掉
cd /opt/graphite/conf
cp aggregation-rules.conf.example aggregation-rules.conf
cp blacklist.conf.example blacklist.conf
cp carbon.conf.example carbon.conf
cp carbon.amqp.conf.example carbon.amqp.conf
cp relay-rules.conf.example relay-rules.conf
cp rewrite-rules.conf.example rewrite-rules.conf
cp storage-schemas.conf.example storage-schemas.conf
cp storage-aggregation.conf.example storage-aggregation.conf
cp whitelist.conf.example whitelist.conf
2、启动carbon-cache:
#cd /opt/graphite/bin
./carbon-cache.py start
1)启动报错:
$ ./carbon-cache.py start
Traceback (most recent call last):
File "./carbon-cache.py", line 28, in <module>
from carbon.util import run_twistd_plugin # noqa
File "/opt/graphite/lib/carbon/util.py", line 15, in <module>
from twisted.python.util import initgroups
ImportError: No module named twisted.python.util
解决:安装twisted
$ pip install twisted
Collecting twisted
Downloading https://files.pythonhosted.org/packages/90/50/4c315ce5d119f67189d1819629cae7908ca0b0a6c572980df5cc6942bc22/Twisted-18.7.0.tar.bz2 (3.1MB)
100% |████████████████████████████████| 3.1MB 133kB/s
Collecting zope.interface>=4.4.2 (from twisted)
Downloading https://files.pythonhosted.org/packages/ac/8a/657532df378c2cd2a1fe6b12be3b4097521570769d4852ec02c24bd3594e/zope.interface-4.5.0.tar.gz (151kB)
100% |████████████████████████████████| 153kB 89kB/s
Collecting constantly>=15.1 (from twisted)
Downloading https://files.pythonhosted.org/packages/b9/65/48c1909d0c0aeae6c10213340ce682db01b48ea900a7d9fce7a7910ff318/constantly-15.1.0-py2.py3-none-any.whl
Collecting incremental>=16.10.1 (from twisted)
Downloading https://files.pythonhosted.org/packages/f5/1d/c98a587dc06e107115cf4a58b49de20b19222c83d75335a192052af4c4b7/incremental-17.5.0-py2.py3-none-any.whl
Collecting Automat>=0.3.0 (from twisted)
Downloading https://files.pythonhosted.org/packages/a3/86/14c16bb98a5a3542ed8fed5d74fb064a902de3bdd98d6584b34553353c45/Automat-0.7.0-py2.py3-none-any.whl
Collecting hyperlink>=17.1.1 (from twisted)
Downloading https://files.pythonhosted.org/packages/a7/b6/84d0c863ff81e8e7de87cff3bd8fd8f1054c227ce09af1b679a8b17a9274/hyperlink-18.0.0-py2.py3-none-any.whl
Collecting PyHamcrest>=1.9.0 (from twisted)
Downloading https://files.pythonhosted.org/packages/9a/d5/d37fd731b7d0e91afcc84577edeccf4638b4f9b82f5ffe2f8b62e2ddc609/PyHamcrest-1.9.0-py2.py3-none-any.whl (52kB)
100% |████████████████████████████████| 61kB 61kB/s
Collecting attrs>=17.4.0 (from twisted)
Downloading https://files.pythonhosted.org/packages/3a/e1/5f9023cc983f1a628a8c2fd051ad19e76ff7b142a0faf329336f9a62a514/attrs-18.2.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools in /usr/lib/python2.7/site-packages (from zope.interface>=4.4.2->twisted) (0.9.8)
Collecting six (from Automat>=0.3.0->twisted)
Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting idna>=2.5 (from hyperlink>=17.1.1->twisted)
Downloading https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl (58kB)
100% |████████████████████████████████| 61kB 19kB/s
Installing collected packages: zope.interface, constantly, incremental, attrs, six, Automat, idna, hyperlink, PyHamcrest, twisted
Running setup.py install for zope.interface ... done
Running setup.py install for twisted ... done
Successfully installed Automat-0.7.0 PyHamcrest-1.9.0 attrs-18.2.0 constantly-15.1.0 hyperlink-18.0.0 idna-2.7 incremental-17.5.0 six-1.11.0 twisted-18.7.0 zope.interface-4.5.0
2)在pip install twisted的时候,有可能网络问题无法下载依赖:
Collecting twisted
Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x3357a50>: Failed to establish a new connection: [Errno 110] Connection timed out',)': /packages/90/50/4c315ce5d119f67189d1819629cae7908ca0b0a6c572980df5cc6942bc22/Twisted-18.7.0.tar.bz2
Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x3357d50>: Failed to establish a new connection: [Errno 110] Connection timed out',)': /packages/90/50/4c315ce5d119f67189d1819629cae7908ca0b0a6c572980df5cc6942bc22/Twisted-18.7.0.tar.bz2
Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x3357f50>: Failed to establish a new connection: [Errno 110] Connection timed out',)': /packages/90/50/4c315ce5d119f67189d1819629cae7908ca0b0a6c572980df5cc6942bc22/Twisted-18.7.0.tar.bz2
Downloading https://files.pythonhosted.org/packages/90/50/4c315ce5d119f67189d1819629cae7908ca0b0a6c572980df5cc6942bc22/Twisted-18.7.0.tar.bz2 (3.1MB)
100% |████████████████████████████████| 3.1MB 5.2MB/s
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-QrXqMw/twisted/setup.py", line 20, in <module>
setuptools.setup(**_setup["getSetupArgs"]())
File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 265, in __init__
self.fetch_build_eggs(attrs.pop('setup_requires'))
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 289, in fetch_build_eggs
parse_requirements(requires), installer=self.fetch_build_egg
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 618, in resolve
dist = best[req.key] = env.best_match(req, self, installer)
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 862, in best_match
return self.obtain(req, installer) # try and download/install
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 874, in obtain
return installer(requirement)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 339, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 611, in easy_install
self.local_index
File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 598, in fetch_distribution
return dist.clone(location=self.download(dist.location, tmpdir))
File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 515, in download
found = self._download_url(scheme.group(1), spec, tmpdir)
File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 762, in _download_url
return self._attempt_download(url, filename)
File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 768, in _attempt_download
headers = self._download_to(url, filename)
File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 662, in _download_to
fp = self.open_url(url)
File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 716, in open_url
% (url, v.reason))
distutils.errors.DistutilsError: Download error for https://files.pythonhosted.org/packages/8f/26/02c4016aa95f45479eea37c90c34f8fab6775732ae62587a874b619ca097/incremental-17.5.0.tar.gz#sha256=7b751696aaf36eebfab537e458929e194460051ccad279c72b755a167eebd4b3: [Errno 101] Network is unreachable
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-QrXqMw/twisted/
解决:根据报错提示连接地址,手动下载incremental-17.5.0.tar.gz,然后python setup.py install
至此,可以成功启动carbon。
$ ./carbon-cache.py start
Starting carbon-cache (instance a)
检查进程:
#netstat -nap | grep 2003
发送数据:
# yum install nc
echo "carbon.agents.graphite-tutorial.metricsReceived 28198 `date +%s`" | nc localhost 2003
3、同步数据:
1)第一种方式:
cd /opt/graphite-web/webapp/
cp manage.py /opt/graphite/webapp/
cd /opt/graphite/webapp/
$ python manage.py syncdb
在执行过程中可以设置用户名密码(用于在web界面上登录)
注
:通过源码安装graphite,manage.py没有被放到/opt/graphite/下,需要手动拷贝过去。
2)第二种方式:
PYTHONPATH=/opt/graphite/webapp django-admin.py migrate --settings=graphite.settings --run-syncdb
Could not import graphite.local_settings, using defaults!
/opt/graphite/webapp/graphite/settings.py:332: UserWarning: SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security
warn('SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security')
Operations to perform:
Synchronize unmigrated apps: functions, render, staticfiles, whitelist, metrics, composer, browser
Apply all migrations: account, sessions, admin, tags, auth, url_shortener, contenttypes, dashboard, events, tagging
Synchronizing apps without migrations:
Creating tables...
Running deferred SQL...
Installing custom SQL...
Running migrations:
Rendering model states... DONE
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying account.0001_initial... OK
Applying admin.0001_initial... OK
Applying contenttypes.0002_remove_content_type_name... OK
Applying auth.0002_alter_permission_name_max_length... OK
Applying auth.0003_alter_user_email_max_length... OK
Applying auth.0004_alter_user_username_opts... OK
Applying auth.0005_alter_user_last_login_null... OK
Applying auth.0006_require_contenttypes_0002... OK
Applying dashboard.0001_initial... OK
Applying events.0001_initial... OK
Applying sessions.0001_initial... OK
Applying tagging.0001_initial... OK
Applying tagging.0002_on_delete... OK
Applying tagging.0003_adapt_max_tag_length... OK
Applying tags.0001_initial... OK
Applying url_shortener.0001_initial... OK
注:执行django-admin.py syncdb –settings=graphite.settings提示找不到syncdb,原因是django 1.7以后需要使用 migrate
四、启动graphite:
graphite指的是一个大的技术集合,她依赖了carbon、whisper、graphite-web三个组件,上面我们已经把carbon启动了。这一步启动graphite其实指的是启动graphite-web,graphite-web是一个python写的web项目,使用了djiango框架。此外,graphite-web除了提供界面外,还提供了很多restful的api接口(比如查询数据、聚合数据等)
1、最简单的方式启动:
# cd /opt/graphite
# PYTHONPATH=`pwd`/storage/whisper ./bin/run-graphite-devel-server.py --port=8085 --libs=`pwd`/webapp /opt/graphite 1>/opt/graphite/storage/log/webapp/process.log 2>&1 &
# tail -f /opt/graphite/storage/log/webapp/process.log
1)可能会遇到如下错误:
Traceback (most recent call last):
File "/usr/lib64/python2.7/wsgiref/handlers.py", line 85, in run
self.result = application(self.environ, self.start_response)
File "/usr/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 189, in __call__
response = self.get_response(request)
File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 218, in get_response
response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 264, in handle_uncaught_exception
if resolver.urlconf_module is None:
File "/usr/lib/python2.7/site-packages/django/core/urlresolvers.py", line 396, in urlconf_module
self._urlconf_module = import_module(self.urlconf_name)
File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/opt/graphite/webapp/graphite/urls.py", line 22, in <module>
url('^render', include('graphite.render.urls')),
File "/usr/lib/python2.7/site-packages/django/conf/urls/__init__.py", line 33, in include
urlconf_module = import_module(urlconf_module)
File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/opt/graphite/webapp/graphite/render/urls.py", line 16, in <module>
from . import views
File "/opt/graphite/webapp/graphite/render/views.py", line 28, in <module>
from graphite.storage import extractForwardHeaders
File "/opt/graphite/webapp/graphite/storage.py", line 27, in <module>
from graphite.readers import MultiReader
File "/opt/graphite/webapp/graphite/readers/__init__.py", line 4, in <module>
from graphite.readers.whisper import WhisperReader, GzippedWhisperReader # noqa # pylint: disable=unused-import
File "/opt/graphite/webapp/graphite/readers/whisper.py", line 9, in <module>
from scandir import scandir, stat # noqa # pylint: disable=unused-import
ImportError: No module named scandir
解决:
pip install scandir
2)再次启动,可能会遇到如下错误:
ImportError: No module named urllib3
解决:
pip install urllib3
3)解决上面依赖后,即可成功启动graphite-web:
我们看启动命令是通过默认的run-graphite-devel-server.py启动python web程序,同时指定了8085端口。在高版本的graphite-web中,通过这种方式启动,在浏览器中输入http:ip:8085,无法访问静态图片、页面等。查看日志,发现404
[17/Sep/2018 03:23:02]"GET / HTTP/1.1" 200 598
[17/Sep/2018 03:23:02]"GET /browser/header HTTP/1.1" 200 1166
[17/Sep/2018 03:23:02]"GET /composer? HTTP/1.1" 200 1643
[17/Sep/2018 03:23:03]"GET /static/img/carbon-fiber.png HTTP/1.1" 404 101
[17/Sep/2018 03:23:03]"GET /static/img/graphite-logo.png HTTP/1.1" 404 102
[17/Sep/2018 03:23:03]"GET /static/js/ext/resources/css/ext-all.css HTTP/1.1" 404 113
[17/Sep/2018 03:23:03]"GET /static/js/browser.js HTTP/1.1" 404 94
[17/Sep/2018 03:23:03]"GET /static/js/composer_widgets.js HTTP/1.1" 404 103
[17/Sep/2018 03:23:03]"GET /static/js/ext/ext-all.js HTTP/1.1" 404 98
[17/Sep/2018 03:23:03]"GET /static/js/ext/adapter/ext/ext-base.js HTTP/1.1" 404 111
[17/Sep/2018 03:23:03]"GET /static/js/composer.js HTTP/1.1" 404 95
[17/Sep/2018 03:23:03]"GET /static/js/completer.js HTTP/1.1" 404 96
[17/Sep/2018 03:23:03]"GET /static/js/ext/ext-all.js HTTP/1.1" 404 98
[17/Sep/2018 03:23:03]"GET /static/js/browser.js HTTP/1.1" 404 94
[17/Sep/2018 03:23:03]"GET /static/js/composer_widgets.js HTTP/1.1" 404 103
[17/Sep/2018 03:23:03]"GET /static/js/composer.js HTTP/1.1" 404 95
[17/Sep/2018 03:23:03]"GET /static/js/completer.js HTTP/1.1" 404 96
我们通过下面命令来收集静态信息:
$ PYTHONPATH=/opt/graphite/webapp django-admin.py collectstatic –noinput –settings=graphite.settings
$ PYTHONPATH=/opt/graphite/webapp django-admin.py collectstatic --noinput --settings=graphite.settings
/opt/graphite/webapp/graphite/settings.py:332: UserWarning: SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security
warn('SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security')
Copying '/opt/graphite/webapp/content/js/browser.js'
Copying '/opt/graphite/webapp/content/js/completer.js'
Copying '/opt/graphite/webapp/content/js/composer.js'
Copying '/opt/graphite/webapp/content/js/composer_widgets.js'
Copying '/opt/graphite/webapp/content/js/dashboard.js'
Copying '/opt/graphite/webapp/content/js/ext/ext-all-debug.js'
Copying '/opt/graphite/webapp/content/js/ext/ext-all.js'
Copying '/opt/graphite/webapp/content/js/ext/resources/images/default/gradient-bg.gif'
....
执行该命令后会在/opt/graphite下生成static目录,里面包含了js、css、html等内容,如下:
$ pwd
/opt/graphite/static
$ ll
drwxr-xr-x 5 root root 35 Sep 17 11:22 admin
drwxr-xr-x 4 root root 156 Sep 17 11:22 css
drwxr-xr-x 2 root root 53 Sep 17 11:22 html
drwxr-xr-x 2 root root 4096 Sep 17 11:22 img
drwxr-xr-x 4 root root 125 Sep 17 11:22 js
执行了这个让然无法访问静态文件,我们通过uwsgi方式来解决。
2、启动方式二:
除了上面的方式启动graphite-web,还可以用下面的方式:
PYTHONPATH=/opt/graphite/webapp/ django-admin.py runserver 10.39.12.24:8085 --settings=graphite.settings
这里使用了djanggo-admin,本质和上面的一样。同样,也是无法访问到静态资源。
3、使用uwsgi+nginx方式启动
:
上面两种方式都是测试环境上来启动的,生产环境大都是Nginx作为web服务器(nginx来部署graphite),这里需要安装下载uwsgi。本人对python不是很熟悉,大致上理解为:上面两种像是通过内置jetty来启动java-web项目;这种方式是通过tomcat部署java-web项目,我们可以通过设置tomcat的参数来优化java-web。
1)安装nginx:
略
2)安装uwsgi:
$ pip install uwsgi
Collecting uwsgi
Downloading https://files.pythonhosted.org/packages/a2/c9/a2d5737f63cd9df4317a4acc15d1ddf4952e28398601d8d7d706c16381e0/uwsgi-2.0.17.1.tar.gz (800kB)
100% |████████████████████████████████| 808kB 739kB/s
Installing collected packages: uwsgi
Running setup.py install for uwsgi ... done
Successfully installed uwsgi-2.0.17.1
3)配置uwsgi文件:
vim /opt/graphite/webapp/graphite.ini(这个文件不存在)
[uwsgi]
post-buffering = 65536
buffer-size = 65536
chdir=/opt/graphite/webapp/
module=graphite.wsgi
master=true
pidfile=/var/run/graphite.pid
vacuum=true
max-requests=20000
daemonize=/opt/graphite/storage/log/webapp/access.log
processes=32
harakiri=10
harakiri-verbose=true
socket=127.0.0.1:9090
stats=127.0.0.1:1716
listen=1024
注意
:在graphite运行一段时间之后可能会出现大量超时的情况,此时需要适当的增加processes数量,但是有一点极其关键需要增加process回收参数设置:harakiri-verbose=true表示超时回收机制,harakiri=10表示超时10即回收
4)配置nginx:
location / {
#root html;
#index index.html index.htm;
uwsgi_pass 127.0.0.1:9090;
include uwsgi_params;
}
location /static/ {
alias /opt/graphite/webapp/content/;
index index.html index.html;
}
5)启动nginx,然后通过http://ip 即可访问到graphite-web界面了。
6)grafana连接graphite:
- 如果通过前两种方式启动的graphite-web,那么在grafana的datasource中设置http://ip:8085;
- 如果通过nginx+uwsgi启动的graphite-web,那么在grafana的DataSource中设置http://ip/ 即可。
注:grafana本质上是通过graphite-web提供的restful-api来获取graphite中carbon存储的数据,然后再展示到界面上。
参考:
https://graphite.readthedocs.io/en/latest/install-source.html
https://blog.csdn.net/weixin_39922154/article/details/79132147
http://www.infoq.com/cn/articles/graphite-intro
https://my.oschina.net/u/1263964/blog/711998
https://uwsgi-docs-zh.readthedocs.io/zh_CN/latest/tutorials/Django_and_nginx.html