This repository was archived by the owner on Aug 29, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 15
Es qat support only #10
Open
ZJie1
wants to merge
24
commits into
Intel-bigdata:master
Choose a base branch
from
ZJie1:ES_QAT_SUPPORT_ONLY
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 17 commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
f0769a6
add common modules, need changes later
ZJie1 d3cc105
add common module, the pom.xml in top update
ZJie1 9d84deb
add es_qat_wrapper modules
ZJie1 996798b
debug the es_qat_wrapper with log
ZJie1 ac62155
[POAE7-10] ES QAT compression support test finished (Zhangjie, Review…
ZJie1 5492c0f
sync the function API with DeflaterCompressor
ZJie1 d3f11b2
remove the LOG to successfully used in ES
ZJie1 c4ad734
add org.apache.logging.log4j to LOG
ZJie1 24552f1
clean code
ZJie1 67d5a5e
[POAE7-10]ES QAT compression clean code finished(Zhangjie, Reviewed b…
ZJie1 cc2d8bd
[POAE7-10]ES QAT compression clean code finished(Zhangjie, Reviewed b…
ZJie1 3a4d88b
add LOG in read and write
ZJie1 282e7b2
[POAE7-10]Add Log in Write and Read to identify QAT(Zhangjie, Reviewe…
ZJie1 bacaeb1
change log level
ZJie1 12d0f87
[POAE7-10](add Hive docs,Zhangjie, Reviewed by XuCheng)
ZJie1 a35f856
[POAE7-10](add Hive docs,Zhangjie, Reviewed by XuCheng)
ZJie1 7852409
add patch of es that with qat and changed lucene
ZJie1 6ddf108
remove unnecessary changes
ZJie1 85996a9
delete unused log dependency in pom
ZJie1 14e02b6
add the apply patches for elasticsearch and lucene with version info
ZJie1 0c64080
add patches of elasticsearch version 7.6.1
ZJie1 c457064
using diff files to apply patches for v7.6.1
ZJie1 87e4e02
delete the version 8.0.0
ZJie1 07c813c
set compression level to 1
ZJie1 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Binary file not shown.
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,215 @@ | ||
| <?xml version="1.0" encoding="UTF-8"?> | ||
| <project xmlns="http://maven.apache.org/POM/4.0.0" | ||
| xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
| xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> | ||
| <parent> | ||
| <artifactId>qat-parent</artifactId> | ||
| <groupId>com.intel.qat</groupId> | ||
| <version>1.0.0</version> | ||
| </parent> | ||
| <modelVersion>4.0.0</modelVersion> | ||
|
|
||
| <artifactId>common</artifactId> | ||
|
|
||
| <properties> | ||
|
|
||
| <!-- Build Properties --> | ||
| <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> | ||
| <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> | ||
| <maven.compiler.useIncrementalCompilation>false</maven.compiler.useIncrementalCompilation> | ||
| <basedir>./</basedir> | ||
|
|
||
| <java.version>1.8</java.version> <!--maybe change--> | ||
|
|
||
| <maven.version>3.3.9</maven.version> | ||
| <maven-compiler-plugin.version>3.6.1</maven-compiler-plugin.version> | ||
| <maven-enforcer-plugin.version>1.4.1</maven-enforcer-plugin.version> | ||
| <maven-surefire-plugin.version>2.19.1</maven-surefire-plugin.version> | ||
| <maven-jar-plugin.version>3.0.2</maven-jar-plugin.version> | ||
| <maven-antrun-plugin.version>1.8</maven-antrun-plugin.version> | ||
| <maven-source-plugin.version>3.0.1</maven-source-plugin.version> | ||
| <maven-clean-plugin.version>3.0.0</maven-clean-plugin.version> | ||
| <maven-javadoc-plugin.version>2.10.4</maven-javadoc-plugin.version> | ||
| <maven-assembly-plugin.version>3.0.0</maven-assembly-plugin.version> | ||
| <maven-shade-plugin.version>3.0.0</maven-shade-plugin.version> | ||
| <maven-install-plugin.version>2.5.2</maven-install-plugin.version> | ||
| <maven-deploy-plugin.version>2.8.2</maven-deploy-plugin.version> | ||
| <maven-dependency-plugin.version>3.0.0</maven-dependency-plugin.version> | ||
| <maven-checkstyle-plugin.version>2.17</maven-checkstyle-plugin.version> | ||
| <junit.version>4.12</junit.version> | ||
|
|
||
| <!-- Modules that copy jars to the build directory should do so under this location. --> | ||
| <jars.target.dir>${project.build.directory}/jars</jars.target.dir> | ||
|
|
||
| <!-- Allow modules to enable / disable certain build plugins easily. --> | ||
| <build.testJarPhase>prepare-package</build.testJarPhase> | ||
| <build.copyDependenciesPhase>package</build.copyDependenciesPhase> | ||
|
|
||
| <CodeCacheSize>512m</CodeCacheSize> | ||
| </properties> | ||
|
|
||
| <repositories> | ||
| <repository> | ||
| <id>mavencentral</id> | ||
| <name>maven central repository</name> | ||
| <url>http://repo1.maven.org/maven2</url> | ||
| <layout>default</layout> | ||
| <releases> | ||
| <enabled>true</enabled> | ||
| <checksumPolicy>warn</checksumPolicy> | ||
| </releases> | ||
| <snapshots> | ||
| <enabled>false</enabled> | ||
| </snapshots> | ||
| </repository> | ||
| <repository> | ||
| <id>datanucleus</id> | ||
| <name>datanucleus maven repository</name> | ||
| <url>http://www.datanucleus.org/downloads/maven2</url> | ||
| <layout>default</layout> | ||
| <releases> | ||
| <enabled>true</enabled> | ||
| <checksumPolicy>warn</checksumPolicy> | ||
| </releases> | ||
| <snapshots> | ||
| <enabled>false</enabled> | ||
| </snapshots> | ||
| </repository> | ||
| <repository> | ||
| <id>glassfish-repository</id> | ||
| <url>http://maven.glassfish.org/content/groups/glassfish</url> | ||
| <releases> | ||
| <enabled>false</enabled> | ||
| </releases> | ||
| <snapshots> | ||
| <enabled>false</enabled> | ||
| </snapshots> | ||
| </repository> | ||
| <repository> | ||
| <id>sonatype-snapshot</id> | ||
| <url>https://oss.sonatype.org/content/repositories/snapshots</url> | ||
| <releases> | ||
| <enabled>false</enabled> | ||
| </releases> | ||
| <snapshots> | ||
| <enabled>false</enabled> | ||
| </snapshots> | ||
| </repository> | ||
| </repositories> | ||
|
|
||
| <pluginRepositories> | ||
| <pluginRepository> | ||
| <id>central</id> | ||
| <url>https://repo1.maven.org/maven2</url> | ||
| <releases> | ||
| <enabled>true</enabled> | ||
| </releases> | ||
| <snapshots> | ||
| <enabled>false</enabled> | ||
| </snapshots> | ||
| </pluginRepository> | ||
| </pluginRepositories> | ||
|
|
||
| <dependencies> | ||
| <dependency> | ||
| <groupId>junit</groupId> | ||
| <artifactId>junit</artifactId> | ||
| <version>${junit.version}</version> | ||
| <scope>test</scope> | ||
| </dependency> | ||
|
|
||
| <!--http://mvnrepository.com/artifact/org.slf4j/slf4j-api--> | ||
| <dependency> | ||
| <groupId>org.slf4j</groupId> | ||
| <artifactId>slf4j-api</artifactId> | ||
| <version>1.7.2</version> | ||
| </dependency> | ||
| <!--https://mvnrepository.com/artifact/org.slf4j/slf4j-log4j12--> | ||
| <dependency> | ||
| <groupId>org.slf4j</groupId> | ||
| <artifactId>slf4j-log4j12</artifactId> | ||
| <version>1.7.2</version> | ||
| <scope>test</scope> | ||
| </dependency> | ||
|
|
||
| </dependencies> | ||
|
|
||
|
|
||
| <build> | ||
| <plugins> | ||
|
|
||
| <plugin> | ||
| <groupId>org.apache.maven.plugins</groupId> | ||
| <artifactId>maven-compiler-plugin</artifactId> | ||
| <version>${maven-compiler-plugin.version}</version> | ||
| <configuration> | ||
| <source>${java.version}</source> | ||
| <target>${java.version}</target> | ||
| <encoding>UTF-8</encoding> | ||
| <maxmem>1024m</maxmem> | ||
| <fork>true</fork> | ||
| <compilerArgs> | ||
| <arg>-Xlint:all,-serial,-path</arg> | ||
| </compilerArgs> | ||
| </configuration> | ||
| </plugin> | ||
| <plugin> | ||
| <groupId>org.apache.maven.plugins</groupId> | ||
| <artifactId>maven-jar-plugin</artifactId> | ||
| <version>${maven-jar-plugin.version}</version> | ||
| <executions> | ||
| <execution> | ||
| <id>prepare-test-jar</id> | ||
| <phase>${build.testJarPhase}</phase> | ||
| <goals> | ||
| <goal>test-jar</goal> | ||
| </goals> | ||
| <configuration> | ||
| <excludes> | ||
| <exclude>log4j.properties</exclude> | ||
| </excludes> | ||
| </configuration> | ||
| </execution> | ||
| </executions> | ||
| </plugin> | ||
| <plugin> | ||
| <groupId>org.apache.maven.plugins</groupId> | ||
| <artifactId>maven-antrun-plugin</artifactId> | ||
| <version>${maven-antrun-plugin.version}</version> | ||
| <executions> | ||
| <execution> | ||
| <id>build-native</id> | ||
| <phase>compile</phase> | ||
| <goals> | ||
| <goal>run</goal> | ||
| </goals> | ||
| <configuration> | ||
| <tasks> | ||
| <property name="native.src.dir" value="${basedir}/src/main/native"/> | ||
| <exec dir="${native.src.dir}" executable="make" failonerror="true"> | ||
| <env key="QATZIPSRC" value="${qatzip.src}" /> | ||
| </exec> | ||
| </tasks> | ||
| </configuration> | ||
| </execution> | ||
| <execution> | ||
| <id>clean-native</id> | ||
| <phase>clean</phase> | ||
| <goals> | ||
| <goal>run</goal> | ||
| </goals> | ||
| <configuration> | ||
| <tasks> | ||
| <exec dir="src/main/native" executable="make" failonerror="true"> | ||
| <arg value="clean" /> | ||
| </exec> | ||
| </tasks> | ||
| </configuration> | ||
| </execution> | ||
| </executions> | ||
| </plugin> | ||
| </plugins> | ||
| </build> | ||
|
|
||
|
|
||
| </project> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one or more | ||
| * contributor license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright ownership. | ||
| * The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| * (the "License"); you may not use this file except in compliance with | ||
| * the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| package com.intel.qat.jni; | ||
|
|
||
| import com.intel.qat.util.NativeCodeLoader; | ||
|
|
||
| import java.nio.ByteBuffer; | ||
|
|
||
| /** | ||
| * JNI bindings to the original C implementation of QatCodec. | ||
| */ | ||
| public enum QatCodecJNI { | ||
| ; | ||
| static { | ||
| NativeCodeLoader.load(); | ||
| init(); | ||
| } | ||
|
|
||
| static native void init(); | ||
| public static native Object allocNativeBuffer(int capacity, int align); | ||
| public static native long createCompressContext(int level); | ||
| public static native long createDecompressContext(); | ||
| public static native void destroyContext(long context); | ||
| public static native int compress(long context, ByteBuffer srcBuffer, int srcOff, int srcLen, | ||
| ByteBuffer destBuffer, int destOff, int maxDestLen); | ||
| public static native int decompress(long context, ByteBuffer srcBuffer, int srcOff, int srcLen, | ||
| ByteBuffer destBuffer, int destOff, int destLen); | ||
| public static native String getLibraryName(int codec); | ||
| } | ||
|
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove these two.