3 ZipKin数据持久化
Zipkin Server默认会将追踪数据信息保存到内存，但这种方式不适合生产环境。Zipkin支持将追踪数据持久化到mysql数据库或elasticsearch中。
使用mysql实现数据持久化
新建一个数据库，名称为zipkin，执行建表语句
建表脚本的官方地址:https://github.com/openzipkin/zipkin/blob/master/zipkin-storage/mysql-v1/src/main/resources/mysql.sql，sql脚本建议从官方地址获取最新。
CREATE TABLE IF NOT EXISTS zipkin_spans (
`trace_id_high` BIGINT NOT NULL DEFAULT 0 COMMENT 'If non zero, this means the trace uses 128 bit traceIds instead of 64 bit',
`trace_id` BIGINT NOT NULL,
`id` BIGINT NOT NULL,
`name` VARCHAR(255) NOT NULL,
`remote_service_name` VARCHAR(255),
`parent_id` BIGINT,
`debug` BIT(1),
`start_ts` BIGINT COMMENT 'Span.timestamp(): epoch micros used for endTs query and to implement TTL',
`duration` BIGINT COMMENT 'Span.duration(): micros used for minDuration and maxDuration query',
PRIMARY KEY (`trace_id_high`, `trace_id`, `id`)
) ENGINE=InnoDB ROW_FORMAT=COMPRESSED CHARACTER SET=utf8 COLLATE utf8_general_ci;
ALTER TABLE zipkin_spans ADD INDEX(`trace_id_high`, `trace_id`) COMMENT 'for getTracesByIds';
ALTER TABLE zipkin_spans ADD INDEX(`name`) COMMENT 'for getTraces and getSpanNames';
ALTER TABLE zipkin_spans ADD INDEX(`remote_service_name`) COMMENT 'for getTraces and getRemoteServiceNames';
ALTER TABLE zipkin_spans ADD INDEX(`start_ts`) COMMENT 'for getTraces ordering and range';
CREATE TABLE IF NOT EXISTS zipkin_annotations (
`trace_id_high` BIGINT NOT NULL DEFAULT 0 COMMENT 'If non zero, this means the trace uses 128 bit traceIds instead of 64 bit',
`trace_id` BIGINT NOT NULL COMMENT 'coincides with zipkin_spans.trace_id',
`span_id` BIGINT NOT NULL COMMENT 'coincides with zipkin_spans.id',
`a_key` VARCHAR(255) NOT NULL COMMENT 'BinaryAnnotation.key or Annotation.value if type == -1',
`a_value` BLOB COMMENT 'BinaryAnnotation.value(), which must be smaller than 64KB',
`a_type` INT NOT NULL COMMENT 'BinaryAnnotation.type() or -1 if Annotation',
`a_timestamp` BIGINT COMMENT 'Used to implement TTL; Annotation.timestamp or zipkin_spans.timestamp',
`endpoint_ipv4` INT COMMENT 'Null when Binary/Annotation.endpoint is null',
`endpoint_ipv6` BINARY(16) COMMENT 'Null when Binary/Annotation.endpoint is null, or no IPv6 address',
`endpoint_port` SMALLINT COMMENT 'Null when Binary/Annotation.endpoint is null',
`endpoint_service_name` VARCHAR(255) COMMENT 'Null when Binary/Annotation.endpoint is null'
) ENGINE=InnoDB ROW_FORMAT=COMPRESSED CHARACTER SET=utf8 COLLATE utf8_general_ci;
ALTER TABLE zipkin_annotations ADD UNIQUE KEY(`trace_id_high`, `trace_id`, `span_id`, `a_key`, `a_timestamp`) COMMENT 'Ignore insert on duplicate';
ALTER TABLE zipkin_annotations ADD INDEX(`trace_id_high`, `trace_id`, `span_id`) COMMENT 'for joining with zipkin_spans';
ALTER TABLE zipkin_annotations ADD INDEX(`trace_id_high`, `trace_id`) COMMENT 'for getTraces/ByIds';
ALTER TABLE zipkin_annotations ADD INDEX(`endpoint_service_name`) COMMENT 'for getTraces and getServiceNames';
ALTER TABLE zipkin_annotations ADD INDEX(`a_type`) COMMENT 'for getTraces and autocomplete values';
ALTER TABLE zipkin_annotations ADD INDEX(`a_key`) COMMENT 'for getTraces and autocomplete values';
ALTER TABLE zipkin_annotations ADD INDEX(`trace_id`, `span_id`, `a_key`) COMMENT 'for dependencies job';
CREATE TABLE IF NOT EXISTS zipkin_dependencies (
`day` DATE NOT NULL,
`parent` VARCHAR(255) NOT NULL,
`child` VARCHAR(255) NOT NULL,
`call_count` BIGINT,
`error_count` BIGINT,
PRIMARY KEY (`day`, `parent`, `child`)
) ENGINE=InnoDB ROW_FORMAT=COMPRESSED CHARACTER SET=utf8 COLLATE utf8_general_ci;
重新启动zipkin server的时候，指定数据保存的mysql信息
java -jar zipkin-server-2.12.9-exec.jar --STORAGE_TYPE=mysql --MYSQL_HOST=127.0.0.1 --MYSQL_TCP_PORT=3306 --MYSQL_DB=zipkin --MYSQL_USER=root --MYSQL_PASS=123456
访问微服务 ， 查看 http://localhost:9411/zipkin/ ，已经可以在第三方访问了
再关闭zipkin ，重新启动发现数据还在
—————————————
3-2 使用elasticsearch实现数据持久化（避免操作受到影响提前删除mysql的zipkin数据库）
由于请求数据应该是比较多的，并且会随着时间的推移，数据会越来越多，因此存储到mysql中就显得非常不合适了，建议将zipkin中的数据持久化到elasticsearch中，并且可以搭建elasticsearch集群，可以很好的对数据进行存储以及以后的查找分析。
解压 elasticsearch-6.8.4.zip =》bin目录 =》双击 elasticsearch.bat 启动
如果要将zipkin中数据持久化到elasticsearch中，那么就在启动zipkin的时候在命令中传入elasticsearch的基本参数：IP和端口号（单机或集群，集群用英文逗号分隔）、index等，还可以传递分片数和副本数等；详细参数见源码包或者jar包的zipkin-server-shared.yml。yml中配置信息如下所示，注意查看elasticsearch相关的参数：
n}
# Consumer group this process is consuming on behalf of.
group-id: ${KAFKA_GROUP_ID:zipkin}
# Count of consumer threads consuming the topic
streams: ${KAFKA_STREAMS:1}
rabbitmq:
enabled: ${COLLECTOR_RABBITMQ_ENABLED:true}
# RabbitMQ server address list (comma-separated list of host:port)
addresses: ${RABBIT_ADDRESSES:}
concurrency: ${RABBIT_CONCURRENCY:1}
# TCP connection timeout in milliseconds
connection-timeout: ${RABBIT_CONNECTION_TIMEOUT:60000}
password: ${RABBIT_PASSWORD:guest}
queue: ${RABBIT_QUEUE:zipkin}
username: ${RABBIT_USER:guest}
virtual-host: ${RABBIT_VIRTUAL_HOST:/}
useSsl: ${RABBIT_USE_SSL:false}
uri: ${RABBIT_URI:}
scribe:
enabled: ${COLLECTOR_SCRIBE_ENABLED:${SCRIBE_ENABLED:false}}
category: ${SCRIBE_CATEGORY:zipkin}
port: ${COLLECTOR_PORT:9410}
query:
enabled: ${QUERY_ENABLED:true}
# 1 day in millis
lookback: ${QUERY_LOOKBACK:86400000}
# The Cache-Control max-age (seconds) for /api/v2/services, /api/v2/remoteServices and /api/v2/spans
names-max-age: 300
# CORS allowed-origins.
allowed-origins: "*"
# Internal properties that end users should never try to use
internal:
actuator:
enabled: true
# auto-configuration to include when ArmeriaSpringActuatorAutoConfiguration is present.
# Note: These are still subject to endpoint conditions. The list must be checked for drift
# upgrading Spring Boot.
include:
- org.springframework.boot.actuate.autoconfigure.beans.BeansEndpointAutoConfiguration
- org.springframework.boot.actuate.autoconfigure.condition.ConditionsReportEndpointAutoConfiguration
- org.springframework.boot.actuate.autoconfigure.context.properties.ConfigurationPropertiesReportEndpointAutoConfiguration
- org.springframework.boot.actuate.autoconfigure.endpoint.EndpointAutoConfiguration
- org.springframework.boot.actuate.autoconfigure.env.EnvironmentEndpointAutoConfiguration
- org.springframework.boot.actuate.autoconfigure.management.HeapDumpWebEndpointAutoConfiguration
- org.springframework.boot.actuate.autoconfigure.logging.LoggersEndpointAutoConfiguration
- org.springframework.boot.actuate.autoconfigure.management.ThreadDumpEndpointAutoConfiguration
storage:
strict-trace-id: ${STRICT_TRACE_ID:true}
search-enabled: ${SEARCH_ENABLED:true}
autocomplete-keys: ${AUTOCOMPLETE_KEYS:}
autocomplete-ttl: ${AUTOCOMPLETE_TTL:3600000}
autocomplete-cardinality: 20000
type: ${STORAGE_TYPE:mem}
throttle:
enabled: ${STORAGE_THROTTLE_ENABLED:false}
min-concurrency: ${STORAGE_THROTTLE_MIN_CONCURRENCY:10}
max-concurrency: ${STORAGE_THROTTLE_MAX_CONCURRENCY:200}
max-queue-size: ${STORAGE_THROTTLE_MAX_QUEUE_SIZE:1000}
mem:
# Maximum number of spans to keep in memory. When exceeded, oldest traces (and their spans) will be purged.
max-spans: ${MEM_MAX_SPANS:500000}
cassandra:
# Comma separated list of host addresses part of Cassandra cluster. Ports default to 9042 but you can also specify a custom port with 'host:port'.
contact-points: ${CASSANDRA_CONTACT_POINTS:localhost}
# Name of the datacenter that will be considered "local" for latency load balancing. When unset, load-balancing is round-robin.
local-dc: ${CASSANDRA_LOCAL_DC:}
# Will throw an exception on startup if authentication fails.
username: ${CASSANDRA_USERNAME:}
password: ${CASSANDRA_PASSWORD:}
keyspace: ${CASSANDRA_KEYSPACE:zipkin}
# Max pooled connections per datacenter-local host.
max-connections: ${CASSANDRA_MAX_CONNECTIONS:8}
# Ensuring that schema exists, if enabled tries to execute script /zipkin-cassandra-core/resources/cassandra-schema.cql.
ensure-schema: ${CASSANDRA_ENSURE_SCHEMA:true}
# 7 days in seconds
span-ttl: ${CASSANDRA_SPAN_TTL:604800}
# 3 days in seconds
index-ttl: ${CASSANDRA_INDEX_TTL:259200}
# the maximum trace index metadata entries to cache
index-cache-max: ${CASSANDRA_INDEX_CACHE_MAX:100000}
# how long to cache index metadata about a trace. 1 minute in seconds
index-cache-ttl: ${CASSANDRA_INDEX_CACHE_TTL:60}
# how many more index rows to fetch than the user-supplied query limit
index-fetch-multiplier: ${CASSANDRA_INDEX_FETCH_MULTIPLIER:3}
# Using ssl for connection, rely on Keystore
use-ssl: ${CASSANDRA_USE_SSL:false}
cassandra3:
# Comma separated list of host addresses part of Cassandra cluster. Ports default to 9042 but you can also specify a custom port with 'host:port'.
contact-points: ${CASSANDRA_CONTACT_POINTS:localhost}
# Name of the datacenter that will be considered "local" for latency load balancing. When unset, load-balancing is round-robin.
local-dc: ${CASSANDRA_LOCAL_DC:}
# Will throw an exception on startup if authentication fails.
username: ${CASSANDRA_USERNAME:}
password: ${CASSANDRA_PASSWORD:}
keyspace: ${CASSANDRA_KEYSPACE:zipkin2}
# Max pooled connections per datacenter-local host.
max-connections: ${CASSANDRA_MAX_CONNECTIONS:8}
# Ensuring that schema exists, if enabled tries to execute script /zipkin2-schema.cql
ensure-schema: ${CASSANDRA_ENSURE_SCHEMA:true}
# how many more index rows to fetch than the user-supplied query limit
index-fetch-multiplier: ${CASSANDRA_INDEX_FETCH_MULTIPLIER:3}
# Using ssl for connection, rely on Keystore
use-ssl: ${CASSANDRA_USE_SSL:false}
elasticsearch:
# host is left unset intentionally, to defer the decision
hosts: ${ES_HOSTS:}
pipeline: ${ES_PIPELINE:}
timeout: ${ES_TIMEOUT:10000}
index: ${ES_INDEX:zipkin}
date-separator: ${ES_DATE_SEPARATOR:-}
index-shards: ${ES_INDEX_SHARDS:5}
index-replicas: ${ES_INDEX_REPLICAS:1}
username: ${ES_USERNAME:}
password: ${ES_PASSWORD:}
http-logging: ${ES_HTTP_LOGGING:}
health-check:
enabled: ${ES_HEALTH_CHECK_ENABLED:true}
interval: ${ES_HEALTH_CHECK_INTERVAL:3s}
mysql:
jdbc-url: ${MYSQL_JDBC_URL:}
host: ${MYSQL_HOST:localhost}
port: ${MYSQL_TCP_PORT:3306}
username: ${MYSQL_USER:}
password: ${MYSQL_PASS:}
db: ${MYSQL_DB:zipkin}
max-active: ${MYSQL_MAX_CONNECTIONS:10}
use-ssl: ${MYSQL_USE_SSL:false}
ui:
enabled: ${QUERY_ENABLED:true}
## Values below here are mapped to ZipkinUiProperties, served as /config.json
# Default limit for Find Traces
query-limit: 10
# The value here becomes a label in the top-right corner
environment:
# Default duration to look back when finding traces.
# Affects the "Start time" element in the UI. 15 minutes in millis
default-lookback: 900000
# When false, disables the "find a trace" screen
search-enabled: ${SEARCH_ENABLED:true}
# Which sites this Zipkin UI covers. Regex syntax. (e.g. http:\/\/example.com\/.*)
# Multiple sites can be specified, e.g.
# - .*example1.com
# - .*example2.com
# Default is "match all websites"
instrumented: .*
# URL placed into the <base> tag in the HTML
base-path: /zipkin
# When false, disables the "Try Lens UI" button in the navigation page
suggest-lens: true
# When true, the classic UI will not be mounted
use-lens: false
# We are using Armeria instead of Tomcat. Have it inherit the default configuration from Spring
spring.main.web-application-type: none
# These defaults are not used directly. They are used via armeria namespacing
server:
port: ${QUERY_PORT:9411}
use-forward-headers: true
compression:
enabled: true
# compresses any response over min-response-size (default is 2KiB)
# Includes dynamic json content and large static assets from zipkin-ui
mime-types: application/json,application/javascript,text/css,image/svg
min-response-size: 2048
armeria:
ports:
- port: ${server.port}
protocols:
- http
compression:
enabled: ${server.compression.enabled}
mime-types: ${server.compression.mime-types}
min-response-size: ${server.compression.min-response-size}
gracefulShutdownQuietPeriodMillis: -1
gracefulShutdownTimeoutMillis: -1
spring:
jmx:
# reduce startup time by excluding unexposed JMX service
enabled: false
mvc:
favicon:
# zipkin has its own favicon
enabled: false
autoconfigure:
# NOTE: These exclusions can drift between Spring Boot minor versions. Audit accordingly.
# Ex. curl -s localhost:9411/actuator/beans|jq '.contexts.application.beans|keys_unsorted[]'|sort
exclude:
# JMX is disabled
- org.springframework.boot.actuate.autoconfigure.endpoint.jmx.JmxEndpointAutoConfiguration
# /health and /actuator/health served directly by Armeria
- org.springframework.boot.actuate.autoconfigure.health.HealthEndpointAutoConfiguration
- org.springframework.boot.actuate.autoconfigure.health.HealthIndicatorAutoConfiguration
# /info and /actuator/info served directly by Armeria (content is /info.json)
- org.springframework.boot.autoconfigure.info.ProjectInfoAutoConfiguration
- org.springframework.boot.actuate.autoconfigure.info.InfoContributorAutoConfiguration
# /prometheus and /actuator/prometheus are served directly by Armeria
- org.springframework.boot.actuate.autoconfigure.metrics.export.prometheus.PrometheusMetricsExportAutoConfiguration
# Remove unused auto-configuration
- org.springframework.boot.autoconfigure.cassandra.CassandraAutoConfiguration
- org.springframework.boot.autoconfigure.jackson.JacksonAutoConfiguration
- org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration
- org.springframework.boot.autoconfigure.jooq.JooqAutoConfiguration
- org.springframework.boot.autoconfigure.jms.activemq.ActiveMQAutoConfiguration
- org.springframework.boot.autoconfigure.task.TaskExecutionAutoConfiguration
- org.springframework.boot.autoconfigure.task.TaskSchedulingAutoConfiguration
logging:
pattern:
level: "%clr(%5p) %clr([%X{traceId}/%X{spanId}]){yellow}"
level:
# Silence ResponseTimeoutException in the Armeria framework since we log it anyways in HTTP
# logging when enabled. https://github.com/line/armeria/issues/2000
com.linecorp.armeria.client.HttpResponseDecoder: 'OFF'
# # investigate /api/v2/dependencies
# zipkin2.internal.DependencyLinker: 'DEBUG'
# # log cassandra queries (DEBUG is without values)
# com.datastax.driver.core.QueryLogger: 'TRACE'
# # log cassandra trace propagation
# com.datastax.driver.core.Message: 'TRACE'
# # log reason behind http collector dropped messages
# zipkin2.server.ZipkinHttpCollector: 'DEBUG'
# zipkin2.collector.kafka.KafkaCollector: 'DEBUG'
# zipkin2.collector.rabbitmq.RabbitMQCollector: 'DEBUG'
# zipkin2.collector.scribe.ScribeCollector: 'DEBUG'
management:
endpoints:
web:
exposure:
include: '*'
# Below are served directly without actuator.
endpoint:
health:
enabled: false
prometheus:
enabled: false
info:
enabled: false
# Disabling auto time http requests since it is added in ZipkinPrometheusMetricsConfiguration
# In Zipkin we use different naming for the http requests duration
metrics:
web:
server:
auto-time-requests: false
Windows上持久化启动命令：
java -jar zipkin-server-2.12.9-exec.jar --STORAGE_TYPE=elasticsearch --ES-HOST=localhost:9200
访问微服务 ， 查看 http://localhost:9411/zipkin/ ，已经可以在第三方访问了
再关闭zipkin ，重新启动发现数据还在
版权声明：本文为qq_44897733原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。