博客

实用技巧

Maven 提速

多线程

# 用 4 个线程构建，以及根据 CPU 核数每个核分配 1 个线程进行构建
$ mvn -T 4 clean install
$ mvn -T 1C clean install

跳过测试

-DskipTests               # 不执行测试用例，但编译测试用例类生成相应的 class 文件至 target/test-classes 下
-Dmaven.test.skip=true    # 不执行测试用例，也不编译测试用例类

# 结合上文的`并行执行`
$ mvn -T 1C clean install -Dmaven.test.skip=true

# 如果还是阻塞: 资源管理器 - shutdown all java app

# 如果 jar 包过大，可以下载按照路径放在 repository 中，之后可能还需要 mvn clean 来下载 groovy-all-2.3.11.pom 文件 (mvnrepository.com)
D:\apps\maven\repository\org\codehaus\groovy\groovy-all\2.3.11\groovy-all-2.3.11.jar

编译失败后，接着编译

# 如果是 Apache Eagle 之类带有几十个子项目的工程，如果从头编译所有的模块，会很耗功夫
# 通过指定之前失败的模块名，可以继续之前的编译
$ mvn -rf :moduleName clean install

跳过失败的模块，编译到最后再报错

$ mvn clean install --fail-at-end

使用 Nexus 本地私服

可参考

Maven 学习（四）：使用 Nexus 搭建 Maven 私服

使用 Aliyun 国内镜像

<mirror>
  <id>alimaven</id>
  <name>aliyun maven</name>
  <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
  <mirrorOf>central</mirrorOf>
</mirror>

指定 Repository 目录

<!-- Default: ~/.m2/repository  -->
<localRepository>D:\apps\maven\repository</localRepository>

如何使用 Maven 编译指定 module

$ mvn install -pl B -am

  -pl, --projects      (Build specified reactor projects instead of all project)
  -am, --also-make     (If project list is specified, also build projects required by the list)
  # And this will build B and the moudules required by B

Maven 标准目录结构

src/main/java             Application/Library sources
src/main/resources        Application/Library resources
src/main/filters          Resource filter files
src/main/webapp           Web application sources
src/test/java             Test sources
src/test/resources        Test resources
src/test/filters          Test resource filter files
src/it                    Integration Tests (primarily for plugins)
src/assembly              Assembly descriptors
src/site                  Site
LICENSE.txt               Project's license
NOTICE.txt                Notices and attributions required by libraries that the project depends on
README.txt                Project's readme

如何在 Maven 中使用多个 source

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>build-helper-maven-plugin</artifactId>
  <executions>
    <execution>
      <phase>generate-sources</phase>
      <goals>
        <goal>add-source</goal>
      </goals>
      <configuration>
        <sources>
          <source>src/main/scala</source>
        </sources>
      </configuration>
    </execution>
  </executions>
</plugin>

Using Scala UnitTest by Maven

安装

Scala

Maven 依赖

<dependency>
  <groupId>org.scalatest</groupId>
  <artifactId>scalatest_2.10</artifactId>
  <version>2.2.1</version>
</dependency>

创建 unittest 需要的 trait

import org.scalatest._
abstract class UnitTestStyle extends FlatSpec
with Matchers with OptionValues with Inside with Inspectors

编写测试

import spark.streaming.detect.SendNetflow
class SendNetflowTest extends UnitTestStyle {
   

  "Clean method" should "output a string" in {

    val s = "0,tcp,http,SF,229,9385,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,9,9,0.00,0.00,0.00,0.00,1.00,0.00,0.00,9,90,1.00,0.00,0.11,0.04,0.00,0.00,0.00,0.00,normal."
    SendNetflow.clean(s) should be("0,229,9385,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,9,9,0.00,0.00,0.00,0.00,1.00,0.00,0.00,9,90,1.00,0.00,0.11,0.04,0.00,0.00,0.00,0.00\tnormal.")
  }

  it should "throw Exception if an empty string is inputted" in {
    val emptyS = ""
    a[RuntimeException] should be thrownBy {
    SendNetflow.clean(emptyS)
  }
}

Tips: Full code is

here

.

Using slf4j by Maven

Maven 配置

<slf4j.version>1.7.12</slf4j.version>
<logback.version>1.1.3</logback.version>

<dependency>
  <groupId>ch.qos.logback</groupId>
  <artifactId>logback-core</artifactId>
  <version>${logback.version}</version>
</dependency>

<dependency>
  <groupId>ch.qos.logback</groupId>
  <artifactId>logback-classic</artifactId>
  <version>${logback.version}</version>
  <exclusions>
    <exclusion>
      <groupId>ch.qos.logback</groupId>
      <artifactId>logback-core</artifactId>
    </exclusion>
    <exclusion>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-api</artifactId>
    </exclusion>
  </exclusions>
</dependency>

<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-api</artifactId>
  <version>${slf4j.version}</version>
</dependency>

<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>jcl-over-slf4j</artifactId>
  <version>${slf4j.version}</version>
  <exclusions>
    <exclusion>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-api</artifactId>
    </exclusion>
  </exclusions>
</dependency>

<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>log4j-over-slf4j</artifactId>
  <version>${slf4j.version}</version>
  <exclusions>
    <exclusion>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-api</artifactId>
    </exclusion>
  </exclusions>
</dependency>

<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>jul-to-slf4j</artifactId>
  <version>${slf4j.version}</version>
  <exclusions>
    <exclusion>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-api</artifactId>
    </exclusion>
  </exclusions>
</dependency>

logback.xml in resources directory

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <!-- %p:Level %m:Message %c.%M:Package+Method %F:%L:File+Line -->
  <property name="pattern" value="%d{yyyy-MM-dd HH:mm:ss.SSS} | %p | %m | %c.%M | %F:%L %n"/>

  <!-- Print in Console -->
  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <encoder charset="UTF-8">
    <pattern>${pattern}</pattern>
    </encoder>
  </appender>

  <root level="ALL">
    <appender-ref