├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── LICENSE.ZSTD ├── Makefile ├── QAT-ZSTD-Plugin-third-party-programs.txt ├── README.md ├── SECURITY.md ├── docs └── images │ └── qatzstdplugin.png ├── qat_zstd_plugin.spec ├── src ├── Makefile ├── qatseqprod.c └── qatseqprod.h └── test ├── Makefile ├── benchmark.c ├── fuzzing ├── Makefile ├── README.md └── qatseqprodfuzzer.c └── test.c /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | We as members, contributors, and leaders pledge to make participation in our 6 | community a harassment-free experience for everyone, regardless of age, body 7 | size, visible or invisible disability, ethnicity, sex characteristics, gender 8 | identity and expression, level of experience, education, socio-economic status, 9 | nationality, personal appearance, race, caste, color, religion, or sexual 10 | identity and orientation. 11 | 12 | We pledge to act and interact in ways that contribute to an open, welcoming, 13 | diverse, inclusive, and healthy community. 14 | 15 | ## Our Standards 16 | 17 | Examples of behavior that contributes to a positive environment for our 18 | community include: 19 | 20 | * Demonstrating empathy and kindness toward other people 21 | * Being respectful of differing opinions, viewpoints, and experiences 22 | * Giving and gracefully accepting constructive feedback 23 | * Accepting responsibility and apologizing to those affected by our mistakes, 24 | and learning from the experience 25 | * Focusing on what is best not just for us as individuals, but for the overall 26 | community 27 | 28 | Examples of unacceptable behavior include: 29 | 30 | * The use of sexualized language or imagery, and sexual attention or advances of 31 | any kind 32 | * Trolling, insulting or derogatory comments, and personal or political attacks 33 | * Public or private harassment 34 | * Publishing others' private information, such as a physical or email address, 35 | without their explicit permission 36 | * Other conduct which could reasonably be considered inappropriate in a 37 | professional setting 38 | 39 | ## Enforcement Responsibilities 40 | 41 | Community leaders are responsible for clarifying and enforcing our standards of 42 | acceptable behavior and will take appropriate and fair corrective action in 43 | response to any behavior that they deem inappropriate, threatening, offensive, 44 | or harmful. 45 | 46 | Community leaders have the right and responsibility to remove, edit, or reject 47 | comments, commits, code, wiki edits, issues, and other contributions that are 48 | not aligned to this Code of Conduct, and will communicate reasons for moderation 49 | decisions when appropriate. 50 | 51 | ## Scope 52 | 53 | This Code of Conduct applies within all community spaces, and also applies when 54 | an individual is officially representing the community in public spaces. 55 | Examples of representing our community include using an official e-mail address, 56 | posting via an official social media account, or acting as an appointed 57 | representative at an online or offline event. 58 | 59 | ## Enforcement 60 | 61 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 62 | reported to the community leaders responsible for enforcement at 63 | CommunityCodeOfConduct AT intel DOT com. 64 | All complaints will be reviewed and investigated promptly and fairly. 65 | 66 | All community leaders are obligated to respect the privacy and security of the 67 | reporter of any incident. 68 | 69 | ## Enforcement Guidelines 70 | 71 | Community leaders will follow these Community Impact Guidelines in determining 72 | the consequences for any action they deem in violation of this Code of Conduct: 73 | 74 | ### 1. Correction 75 | 76 | **Community Impact**: Use of inappropriate language or other behavior deemed 77 | unprofessional or unwelcome in the community. 78 | 79 | **Consequence**: A private, written warning from community leaders, providing 80 | clarity around the nature of the violation and an explanation of why the 81 | behavior was inappropriate. A public apology may be requested. 82 | 83 | ### 2. Warning 84 | 85 | **Community Impact**: A violation through a single incident or series of 86 | actions. 87 | 88 | **Consequence**: A warning with consequences for continued behavior. No 89 | interaction with the people involved, including unsolicited interaction with 90 | those enforcing the Code of Conduct, for a specified period of time. This 91 | includes avoiding interactions in community spaces as well as external channels 92 | like social media. Violating these terms may lead to a temporary or permanent 93 | ban. 94 | 95 | ### 3. Temporary Ban 96 | 97 | **Community Impact**: A serious violation of community standards, including 98 | sustained inappropriate behavior. 99 | 100 | **Consequence**: A temporary ban from any sort of interaction or public 101 | communication with the community for a specified period of time. No public or 102 | private interaction with the people involved, including unsolicited interaction 103 | with those enforcing the Code of Conduct, is allowed during this period. 104 | Violating these terms may lead to a permanent ban. 105 | 106 | ### 4. Permanent Ban 107 | 108 | **Community Impact**: Demonstrating a pattern of violation of community 109 | standards, including sustained inappropriate behavior, harassment of an 110 | individual, or aggression toward or disparagement of classes of individuals. 111 | 112 | **Consequence**: A permanent ban from any sort of public interaction within the 113 | community. 114 | 115 | ## Attribution 116 | 117 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], 118 | version 2.1, available at 119 | [https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1]. 120 | 121 | Community Impact Guidelines were inspired by 122 | [Mozilla's code of conduct enforcement ladder][Mozilla CoC]. 123 | 124 | For answers to common questions about this code of conduct, see the FAQ at 125 | [https://www.contributor-covenant.org/faq][FAQ]. Translations are available at 126 | [https://www.contributor-covenant.org/translations][translations]. 127 | 128 | [homepage]: https://www.contributor-covenant.org 129 | [v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html 130 | [Mozilla CoC]: https://github.com/mozilla/diversity 131 | [FAQ]: https://www.contributor-covenant.org/faq 132 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing 2 | 3 | ### License 4 | 5 | is licensed under the terms in [LICENSE]. By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms. 6 | 7 | ### Sign your work 8 | 9 | Please use the sign-off line at the end of the patch. Your signature certifies that you wrote the patch or otherwise have the right to pass it on as an open-source patch. The rules are pretty simple: if you can certify 10 | the below (from [developercertificate.org](http://developercertificate.org/)): 11 | 12 | ``` 13 | Developer Certificate of Origin 14 | Version 1.1 15 | 16 | Copyright (C) 2004, 2006 The Linux Foundation and its contributors. 17 | 660 York Street, Suite 102, 18 | San Francisco, CA 94110 USA 19 | 20 | Everyone is permitted to copy and distribute verbatim copies of this 21 | license document, but changing it is not allowed. 22 | 23 | Developer's Certificate of Origin 1.1 24 | 25 | By making a contribution to this project, I certify that: 26 | 27 | (a) The contribution was created in whole or in part by me and I 28 | have the right to submit it under the open source license 29 | indicated in the file; or 30 | 31 | (b) The contribution is based upon previous work that, to the best 32 | of my knowledge, is covered under an appropriate open source 33 | license and I have the right under that license to submit that 34 | work with modifications, whether created in whole or in part 35 | by me, under the same open source license (unless I am 36 | permitted to submit under a different license), as indicated 37 | in the file; or 38 | 39 | (c) The contribution was provided directly to me by some other 40 | person who certified (a), (b) or (c) and I have not modified 41 | it. 42 | 43 | (d) I understand and agree that this project and the contribution 44 | are public and that a record of the contribution (including all 45 | personal information I submit with it, including my sign-off) is 46 | maintained indefinitely and may be redistributed consistent with 47 | this project or the open source license(s) involved. 48 | ``` 49 | 50 | Then you just add a line to every git commit message: 51 | 52 | Signed-off-by: Joe Smith 53 | 54 | Use your real name (sorry, no pseudonyms or anonymous contributions.) 55 | 56 | If you set your `user.name` and `user.email` git configs, you can sign your 57 | commit automatically with `git commit -s`. 58 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | ################################################################ 2 | # BSD LICENSE 3 | # 4 | # Copyright(c) 2007-2023 Intel Corporation. All rights reserved. 5 | # All rights reserved. 6 | # 7 | # Redistribution and use in source and binary forms, with or without 8 | # modification, are permitted provided that the following conditions 9 | # are met: 10 | # 11 | # * Redistributions of source code must retain the above copyright 12 | # notice, this list of conditions and the following disclaimer. 13 | # * Redistributions in binary form must reproduce the above copyright 14 | # notice, this list of conditions and the following disclaimer in 15 | # the documentation and/or other materials provided with the 16 | # distribution. 17 | # * Neither the name of Intel Corporation nor the names of its 18 | # contributors may be used to endorse or promote products derived 19 | # from this software without specific prior written permission. 20 | # 21 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 22 | # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 23 | # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 24 | # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 25 | # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 26 | # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 27 | # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 28 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 29 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 30 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 31 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 32 | # 33 | ################################################################ 34 | -------------------------------------------------------------------------------- /LICENSE.ZSTD: -------------------------------------------------------------------------------- 1 | BSD License 2 | 3 | For Zstandard software 4 | 5 | Copyright (c) Meta Platforms, Inc. and affiliates. All rights reserved. 6 | 7 | Redistribution and use in source and binary forms, with or without modification, 8 | are permitted provided that the following conditions are met: 9 | 10 | * Redistributions of source code must retain the above copyright notice, this 11 | list of conditions and the following disclaimer. 12 | 13 | * Redistributions in binary form must reproduce the above copyright notice, 14 | this list of conditions and the following disclaimer in the documentation 15 | and/or other materials provided with the distribution. 16 | 17 | * Neither the name Facebook, nor Meta, nor the names of its contributors may 18 | be used to endorse or promote products derived from this software without 19 | specific prior written permission. 20 | 21 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 22 | ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 23 | WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 24 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR 25 | ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 26 | (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 27 | LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON 28 | ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 29 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 30 | SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 31 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | # ####################################################################### 2 | # 3 | # BSD LICENSE 4 | # 5 | # Copyright(c) 2007-2023 Intel Corporation. All rights reserved. 6 | # All rights reserved. 7 | # 8 | # Redistribution and use in source and binary forms, with or without 9 | # modification, are permitted provided that the following conditions 10 | # are met: 11 | # 12 | # * Redistributions of source code must retain the above copyright 13 | # notice, this list of conditions and the following disclaimer. 14 | # * Redistributions in binary form must reproduce the above copyright 15 | # notice, this list of conditions and the following disclaimer in 16 | # the documentation and/or other materials provided with the 17 | # distribution. 18 | # * Neither the name of Intel Corporation nor the names of its 19 | # contributors may be used to endorse or promote products derived 20 | # from this software without specific prior written permission. 21 | # 22 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 23 | # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 24 | # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 25 | # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 26 | # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 27 | # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 28 | # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 29 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 30 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 31 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 32 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 33 | # 34 | # ####################################################################### 35 | 36 | SRCDIR = src 37 | TESTDIR = test 38 | 39 | .PHONY: default 40 | default: lib 41 | 42 | .PHONY: lib 43 | lib: 44 | $(Q)$(MAKE) -C $(SRCDIR) $@ 45 | 46 | .PHONY: test 47 | test: 48 | $(Q)$(MAKE) -C $(TESTDIR) $@ 49 | 50 | .PHONY: benchmark 51 | benchmark: 52 | $(Q)$(MAKE) -C $(TESTDIR) $@ 53 | 54 | .PHONY: install 55 | install: 56 | $(Q)$(MAKE) -C $(SRCDIR) $@ 57 | 58 | .PHONY: uninstall 59 | uninstall: 60 | $(Q)$(MAKE) -C $(SRCDIR) $@ 61 | 62 | clean: 63 | $(Q)$(MAKE) -C $(SRCDIR) $@ 64 | $(Q)$(MAKE) -C $(TESTDIR) $@ 65 | 66 | ######################## 67 | # RPM package building # 68 | ######################## 69 | rpm: 70 | mkdir -p rpmbuild/BUILD rpmbuild/RPMS rpmbuild/SOURCES rpmbuild/SPECS rpmbuild/SRPMS 71 | rpmbuild --undefine=_disable_source_fetch --define "_topdir $(PWD)/rpmbuild" -ba qat_zstd_plugin.spec 72 | 73 | rpmclean: 74 | @rm -fr rpmbuild 75 | 76 | .PHONY: rpm rpmclean 77 | -------------------------------------------------------------------------------- /QAT-ZSTD-Plugin-third-party-programs.txt: -------------------------------------------------------------------------------- 1 | QAT ZSTD Plugin Third Party Programs File 2 | 3 | This file contains the list of third party software (“third party programs”) contained in the Intel software and their required notices and/or license terms. This third party software, even if included with the distribution of the Intel software, may be governed by separate license terms, including without limitation, third party license terms, other Intel software license terms, and open source software license terms. These separate license terms govern your use of the third party programs as set forth in the “third-party-programs.txt” or other similarly-named text file. 4 | 5 | Third party programs and their corresponding required notices and/or license terms are listed below. 6 | 7 | ------------------------------------------------------------- 8 | 1. Software Released under the BSD License: 9 | 10 | Zstandard 11 | Copyright (c) Meta Platforms, Inc. and affiliates. 12 | 13 | BSD License 14 | 15 | Redistribution and use in source and binary forms, with or without modification, 16 | are permitted provided that the following conditions are met: 17 | 18 | * Redistributions of source code must retain the above copyright notice, this 19 | list of conditions and the following disclaimer. 20 | 21 | * Redistributions in binary form must reproduce the above copyright notice, 22 | this list of conditions and the following disclaimer in the documentation 23 | and/or other materials provided with the distribution. 24 | 25 | * Neither the name Facebook, nor Meta, nor the names of its contributors may 26 | be used to endorse or promote products derived from this software without 27 | specific prior written permission. 28 | 29 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 30 | ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 31 | WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 32 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR 33 | ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 34 | (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 35 | LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON 36 | ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 37 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 38 | SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 39 | 40 | ------------------------------------------------------------- 41 | Other names and brands may be claimed as the property of others. 42 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Intel® QuickAssist Technology ZSTD Plugin (QAT ZSTD Plugin) 2 | 3 | ## Table of Contents 4 | 5 | - [Introduction](#introduction) 6 | - [Licensing](#licensing) 7 | - [Hardware Requirements](#hardware-requirements) 8 | - [Software Requirements](#software-requirements) 9 | - [Limitations](#limitations) 10 | - [Installation Instructions](#installation-instructions) 11 | - [Legal](#legal) 12 | 13 | ## Introduction 14 | 15 | Intel® QuickAssist Technology ZSTD Plugin (QAT ZSTD Plugin) is a plugin to Zstandard*(ZSTD*) for accelerating compression by QAT. ZSTD* is a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression ratios. ZSTD* provides block-level sequence producer API which allows users to register their custom sequence producer that libzstd invokes to process each block from [1.5.4][1]. The produced list of sequences (literals and matches) is then post-processed by libzstd to produce valid compressed blocks. 16 | 17 | Intel® QuickAssist Technology (Intel® QAT) provides cryptographic and compression acceleration capabilities used to improve performance and efficiency across the data center. QAT sequence producer will offload the process of producing block-level sequences of L1-L12 compression to Intel® QAT, and get performance gain. 18 | 19 |

20 | drawing 21 |

22 | 23 | ## Licensing 24 | 25 | The Licensing of the files within this project is split as follows: 26 | 27 | Intel® QuickAssist Technology ZSTD Plugin - BSD License. Please see the `LICENSE` file contained in the top level folder. Further details can be found in the file headers of the relevant files. 28 | 29 | ## Hardware Requirements 30 | 31 | Intel® 4xxx (Intel® QuickAssist Technology Gen 4) 32 | 33 | ## Software Requirements 34 | 35 | ZSTD* library of version 1.5.4+ 36 | 37 | [public ZSTD-lib][11] 38 | 39 | [Intel® QAT Driver for Linux* Hardware v2.0][2] or [Intel® QuickAssist Technology Library (QATlib)][3] of version 22.07.0+ 40 | 41 | ## Limitations 42 | 43 | 1. Supports compression levels L1 to L12 44 | 2. ZSTD* sequence producer only supports ZSTD* compression API which respects advanced parameters, such as `ZSTD_compress2`, `ZSTD_compressStream2`. 45 | 3. The ZSTD_c_enableLongDistanceMatching cParam is not currently supported. Compression will fail if it is enabled and tries to compress with QAT sequence producer. 46 | 4. Dictionaries are not currently supported. Compression will succeed if the dictionary is referenced, but the dictionary will have no effect. 47 | 5. Stream history is not currently supported. All advanced ZSTD* compression APIs, including streaming APIs, work with QAT sequence producer, but each block is treated as an independent chunk without history from previous blocks. 48 | 6. Multi-threading within a single compression is not currently supported. In other words, compression will fail if `ZSTD_c_nbWorkers` > 0 and an external sequence producer is registered. Each thread must have its own context (CCtx). 49 | 50 | For more details about ZSTD* sequence producer, please refer to [zstd.h][4]. 51 | 52 | ## Installation Instructions 53 | 54 | ### Build and install Intel® QuickAssist Technology Driver 55 | 56 | Users can choose [Intel® QAT Driver for Linux* Hardware v2.0][2](out-of-tree) or [Intel® QuickAssist Technology Library (QATlib)][3](in-tree) according to their requirements. 57 | 58 | If using out-of-tree driver, the user needs to set `ICP_ROOT` environment variable: 59 | 60 | `ICP_ROOT`: the root directory of the QAT driver source tree 61 | 62 | #### Build and install Intel® QAT Driver for Linux* Hardware v2.0 63 | 64 | Download from [Intel® QAT Driver for Linux* Hardware v2.0][2], follow the guidance: [Intel® QuickAssist Technology Software for Linux* - Getting Started Guide][5]. 65 | 66 | If installing the Intel® QAT 2.0 driver for use in a virtual environment, please refer to [Using Intel® Virtualization Technology (Intel® VT) with Intel® QuickAssist Technology][6] 67 | 68 | After installing the QAT driver, please refer to [Intel® QuickAssist Technology Software for Linux* - Programmer's Guide][7] to the update QAT configuration file according to requirements. 69 | 70 | QAT ZSTD Plugin needs a [SHIM] section by default. 71 | There are two ways to change: 72 | * QAT driver default conf file does not contain a [SHIM] section which the QAT ZSTD Plugin requires by default. You can add a [SHIM] section for QAT ZSTD Plugin. 73 | * The default section name in the QAT ZSTD Plugin can be modified if required by setting the environment variable "QAT_SECTION_NAME". 74 | 75 | After updating the configuration files, please restart QAT. 76 | 77 | ```bash 78 | service qat_service restart 79 | ``` 80 | 81 | #### Install QATlib 82 | 83 | QATlib has been upstream to some platforms, RedHat, SUSE. Users also can install QATlib from source code according to [qatlib/INSTALL][8]. 84 | 85 | ### Build QAT sequence producer library 86 | 87 | Before build, set the QATlib environment variables to prevent compilation errors. 88 | 89 | ```bash 90 | export LIBRARY_PATH=/usr/local/lib 91 | export LD_LIBRARY_PATH=/usr/local/lib 92 | ``` 93 | 94 | Shared Virtual Memory (SVM) allows direct submission of an application buffer, thus removing the memcpy cycle cost, cache thrashing, and memory bandwidth. The SVM feature enables passing virtual addresses to the QAT hardware for processing acceleration requests. 95 | 96 | QAT sequence producer library runs on the SVM environment by default. 97 | 98 | To enable SVM, please refer to [Using Intel® Virtualization Technology (Intel® VT) with Intel® QuickAssist Technology][6] to update the BIOS and [Intel® QuickAssist Technology Software for Linux* - Programmer's Guide][7] to update driver configuration. 99 | 100 | ```bash 101 | make 102 | ``` 103 | 104 | If ZSTD* 1.5.4 library is not installed, need to specify path to ZSTD* lib source root by compile variable "ZSTDLIB". 105 | 106 | ```bash 107 | make ZSTDLIB=[PATH TO ZSTD LIB SOURCE] 108 | ``` 109 | 110 | ### USDM support 111 | 112 | If SVM is not enabled, memory passed to Intel® QuickAssist Technology hardware must be DMA enabled. 113 | 114 | Intel provides a User Space DMA-able Memory (USDM) component (kernel driver and corresponding user space library) which allocates/frees DMA-able memory, mapped to user space, performs virtual to physical address translation on memory allocated by this library. Please refer to [Intel® QuickAssist Technology Software for Linux* - Programmer's Guide][7]. 115 | 116 | QAT ZSTD Plugin will automatically switch to USDM mode when SVM is not enabled. 117 | 118 | ### Build and run test program 119 | 120 | ```bash 121 | make test 122 | ./test/test [TEST FILENAME] 123 | ``` 124 | 125 | ### Build and run benchmark tool 126 | 127 | The `benchmark` is a tool used to perform QAT sequence producer performance tests, it supports the following options: 128 | 129 | ```bash 130 | -t# Set maximum threads [1 - 128] (default: 1) 131 | -l# Set iteration loops [1 - 1000000](default: 1) 132 | -c# Set chunk size (default: 32K) 133 | -E# Auto/enable/disable searchForExternalRepcodes(0: auto; 1: enable; 2: disable; default: auto) 134 | -L# Set compression level [1 - 12] (default: 1) 135 | -m# Benchmark mode, 0: software compression; 1:QAT compression(default: 1) 136 | ``` 137 | 138 | In order to get a better performance, increasing the number of threads with `-t` is a better way. The number of dc instances provided by Intel® QAT needs to be increased while increasing test threads, it can be increased by modifying the `NumberDcInstances` in `/etc/4xxx_devx.conf`. Note that the test threads number should not exceed the number of dc instances, as this ensures that each test thread can obtain a dc instance. 139 | For more Intel® QAT configuration information, please refer to [Intel® QuickAssist Technology Software for Linux* - Programmer's Guide][7]. 140 | An example usage of benchmark tool with [Silesia compression corpus][9]: 141 | 142 | Silesia is standard lossless data compression corpora. 143 | 144 | ```bash 145 | ./benchmark -m1 -l100 -c64K -t64 -E2 Silesia 146 | ``` 147 | 148 | which used the following Intel® QAT configuration file: 149 | 150 | ```bash 151 | # QAT configuration file /etc/4xxx_devx.conf 152 | ############################################## 153 | # User Process Instance Section 154 | ############################################## 155 | [SHIM] 156 | NumberCyInstances = 0 157 | NumberDcInstances = 64 158 | NumProcesses = 1 159 | LimitDevAccess = 0 160 | 161 | # Data Compression - User instance #0 162 | Dc1Name = "Dc0" 163 | Dc1IsPolled = 1 164 | # List of core affinities 165 | Dc1CoreAffinity = 0 166 | 167 | # Data Compression - User instance #1 168 | Dc2Name = "Dc1" 169 | Dc2IsPolled = 1 170 | # List of core affinities 171 | Dc2CoreAffinity = 1 172 | ... 173 | # Data Compression - User instance #63 174 | Dc63Name = "Dc63" 175 | Dc63IsPolled = 1 176 | # List of core affinities 177 | Dc63CoreAffinity = 63 178 | ``` 179 | 180 | ### How to integrate QAT sequence producer into `zstd` 181 | Integrating QAT sequence producer into the `zstd` command can speed up its compression, The following sample code shows how to enable QAT sequence producer by modifying the code of `FIO_compressZstdFrame` in `zstd/programs/fileio.c`, including qatseqprod.h in fileio.c and adding -lqatseqprod into Makefile. 182 | 183 | Start QAT device and register qatSequenceProducer before starting compression job. 184 | 185 | ```c 186 | /* Start QAT device, start QAT device at any 187 | time before compression job started */ 188 | QZSTD_startQatDevice(); 189 | /* Create sequence producer state for QAT sequence producer */ 190 | void *sequenceProducerState = QZSTD_createSeqProdState(); 191 | /* register qatSequenceProducer */ 192 | ZSTD_registerSequenceProducer( 193 | ress.cctx, 194 | sequenceProducerState, 195 | qatSequenceProducer 196 | ); 197 | /* Enable sequence producer fallback */ 198 | ZSTD_CCtx_setParameter(ress.cctx, ZSTD_c_enableSeqProducerFallback, 1); 199 | ``` 200 | 201 | Stop QAT device after compression job 202 | 203 | ```c 204 | /* Free sequence producer state */ 205 | QZSTD_freeSeqProdState(sequenceProducerState); 206 | /* Please call QZSTD_stopQatDevice before 207 | QAT is no longer used or the process exits */ 208 | QZSTD_stopQatDevice(); 209 | ``` 210 | 211 | Then recompile `zstd` with flag `-lqatseqprod`. Currently, only single-threaded mode compression is supported to using QAT sequence producer, please run `zstd` with the `--single-thread`. 212 | 213 | Note : some parameters of `zstd` do not support sequence producer, for more zstd usage information please refer to [zstd manual][10]. 214 | 215 | ```bash 216 | ./zstd --single-thread [TEST FILENAME] 217 | ``` 218 | 219 | ### How to integrate QAT sequence producer into applications 220 | 221 | **Initialization** 222 | 223 | Start and initialize the QAT device. 224 | 225 | Create sequence producer state for QAT sequence producer, then call `ZSTD_registerSequenceProducer` to register it in the application source code. 226 | 227 | ```c 228 | ZSTD_CCtx* const zc = ZSTD_createCCtx(); 229 | /* Start QAT device, start QAT device at any 230 | time before compression job started */ 231 | QZSTD_startQatDevice(); 232 | /* Create sequence producer state for QAT sequence producer */ 233 | void *sequenceProducerState = QZSTD_createSeqProdState(); 234 | /* register qatSequenceProducer */ 235 | ZSTD_registerSequenceProducer( 236 | zc, 237 | sequenceProducerState, 238 | qatSequenceProducer 239 | ); 240 | /* Enable sequence producer fallback */ 241 | ZSTD_CCtx_setParameter(zc, ZSTD_c_enableSeqProducerFallback, 1); 242 | ``` 243 | 244 | **Compression API** 245 | 246 | No changes to the application with calling ZSTD* compression API, keep calling `ZSTD_compress2`, `ZSTD_compressStream2`, or `ZSTD_compressStream` to compress. 247 | 248 | ```c 249 | /* Compress */ 250 | ZSTD_compress2(zc, dstBuffer, dstBufferSize, srcBuffer, srcbufferSize); 251 | ``` 252 | 253 | **Free resources and shutdown QAT device** 254 | 255 | ```c 256 | /* Free sequence producer state */ 257 | QZSTD_freeSeqProdState(sequenceProducerState); 258 | /* Please call QZSTD_stopQatDevice before 259 | QAT is no longer used or the process exits */ 260 | QZSTD_stopQatDevice(); 261 | ``` 262 | 263 | Then link to libzstd and libqatseqprod like test program did. 264 | See the DEMO in test/test.c file 265 | 266 | ## Legal 267 | 268 | Intel® disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade. 269 | 270 | This document contains information on products, services and/or processes in development. All information provided here is subject to change without notice. Contact your Intel® representative to obtain the latest forecast, schedule, specifications and roadmaps. 271 | 272 | The products and services described may contain defects or errors known as errata which may cause deviations from published specifications. Current characterized errata are available on request. 273 | 274 | Copies of documents which have an order number and are referenced in this document may be obtained by calling 1-800-548-4725 or by visiting www.intel.com/design/literature.htm. 275 | 276 | Intel, the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries. 277 | 278 | \*Other names and brands may be claimed as the property of others 279 | 280 | [1]:https://github.com/facebook/zstd/releases/tag/v1.5.4 281 | [2]:https://www.intel.com/content/www/us/en/download/765501.html 282 | [3]:https://github.com/intel/qatlib 283 | [4]:https://github.com/facebook/zstd/blob/dev/lib/zstd.h 284 | [5]:https://www.intel.com/content/www/us/en/content-details/632506/intel-quickassist-technology-intel-qat-software-for-linux-getting-started-guide-hardware-version-2-0.html 285 | [6]:https://www.intel.com/content/www/us/en/content-details/709210/using-intel-virtualization-technology-intel-vt-with-intel-quickassist-technology-application-note.html 286 | [7]:https://intel.github.io/quickassist/PG/index.html 287 | [8]:https://github.com/intel/qatlib/blob/main/INSTALL 288 | [9]:https://sun.aei.polsl.pl//~sdeor/index.php?page=silesia 289 | [10]:https://github.com/facebook/zstd/blob/dev/doc/zstd_manual.html 290 | [11]:https://github.com/facebook/zstd 291 | -------------------------------------------------------------------------------- /SECURITY.md: -------------------------------------------------------------------------------- 1 | # Security Policy 2 | Intel is committed to rapidly addressing security vulnerabilities affecting our customers and providing clear guidance on the solution, impact, severity and mitigation. 3 | 4 | ## Reporting a Security Vulnerability 5 | 6 | Please report any security vulnerabilities in this project utilizing the guidelines [here](https://www.intel.com/content/www/us/en/security-center/vulnerability-handling-guidelines.html). 7 | -------------------------------------------------------------------------------- /docs/images/qatzstdplugin.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/intel/QAT-ZSTD-Plugin/3ef18c7cf122539613c1d956670f744864b8a0bc/docs/images/qatzstdplugin.png -------------------------------------------------------------------------------- /qat_zstd_plugin.spec: -------------------------------------------------------------------------------- 1 | Name: qat-zstd-plugin 2 | Version: 0.2.0 3 | Release: %autorelease 4 | Summary: Intel QuickAssist Technology ZSTD Plugin 5 | 6 | License: BSD-3-Clause 7 | URL: https://github.com/intel/QAT-ZSTD-Plugin 8 | Source0: %{url}/archive/v%{version}/%{name}-%{version}.tar.gz 9 | 10 | BuildRequires: automake 11 | BuildRequires: gcc 12 | BuildRequires: libzstd-devel 13 | BuildRequires: qatlib-devel 14 | 15 | # Upstream only supports x86_64 16 | ExclusiveArch: x86_64 17 | 18 | %description 19 | Intel QuickAssist Technology ZSTD is a plugin to Zstandard for accelerating 20 | compression by QAT. ZSTD* is a fast lossless compression algorithm, targeting 21 | real-time compression scenarios at zlib-level and better compression ratios. 22 | 23 | %package devel 24 | Summary: Headers and libraries of QAT-ZSTD-Plugin 25 | Requires: %{name}%{?_isa} = %{version}-%{release} 26 | 27 | %description devel 28 | This package contains headers and libraries required to build applications 29 | that use the QAT ZSTD Plugin. 30 | 31 | %prep 32 | %autosetup -p1 -n QAT-ZSTD-Plugin-%{version} 33 | 34 | %build 35 | %make_build 36 | make 37 | 38 | %install 39 | make install LIBDIR=%{buildroot}%{_libdir} INCLUDEDIR=%{buildroot}%{_includedir} 40 | rm %{buildroot}%{_libdir}/libqatseqprod.a 41 | 42 | %files 43 | %license LICENSE* 44 | %{_libdir}/libqatseqprod.so 45 | 46 | %files devel 47 | %{_libdir}/libqatseqprod.so 48 | %{_includedir}/qatseqprod.h 49 | 50 | %changelog 51 | * Tue Jul 30 2024 Zhu Chengfei - 0.2.0-1 52 | - Update to qat-zstd-plugin v0.2.0 53 | -------------------------------------------------------------------------------- /src/Makefile: -------------------------------------------------------------------------------- 1 | # ####################################################################### 2 | # 3 | # BSD LICENSE 4 | # 5 | # Copyright(c) 2007-2023 Intel Corporation. All rights reserved. 6 | # All rights reserved. 7 | # 8 | # Redistribution and use in source and binary forms, with or without 9 | # modification, are permitted provided that the following conditions 10 | # are met: 11 | # 12 | # * Redistributions of source code must retain the above copyright 13 | # notice, this list of conditions and the following disclaimer. 14 | # * Redistributions in binary form must reproduce the above copyright 15 | # notice, this list of conditions and the following disclaimer in 16 | # the documentation and/or other materials provided with the 17 | # distribution. 18 | # * Neither the name of Intel Corporation nor the names of its 19 | # contributors may be used to endorse or promote products derived 20 | # from this software without specific prior written permission. 21 | # 22 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 23 | # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 24 | # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 25 | # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 26 | # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 27 | # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 28 | # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 29 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 30 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 31 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 32 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 33 | # 34 | # ####################################################################### 35 | 36 | .PHONY: default 37 | default: lib 38 | 39 | DESTDIR ?= 40 | PREFIX ?= /usr/local 41 | LIBDIR ?= $(PREFIX)/lib 42 | INCLUDEDIR ?= $(PREFIX)/include 43 | 44 | CP ?= cp 45 | INSTALL ?= install 46 | 47 | INSTALL_PROGRAM ?= $(INSTALL) 48 | INSTALL_DATA ?= $(INSTALL) -m 644 49 | 50 | ifneq ($(ICP_ROOT), ) 51 | QATFLAGS = -I$(ICP_ROOT)/quickassist/include \ 52 | -I$(ICP_ROOT)/quickassist/include/dc \ 53 | -I$(ICP_ROOT)/quickassist/lookaside/access_layer/include \ 54 | -I$(ICP_ROOT)/quickassist/utilities/libusdm_drv 55 | LDFLAGS = -Wl,-rpath,$(ICP_ROOT)/build -L$(ICP_ROOT)/build -lqat_s \ 56 | -lusdm_drv_s 57 | else 58 | QATFLAGS = -DINTREE 59 | LDFLAGS = -lqat -lusdm 60 | endif 61 | 62 | ifdef ZSTDLIB 63 | CFLAGS += -I$(ZSTDLIB) 64 | endif 65 | 66 | CFLAGS += -Wall -Werror -Wextra -Wcast-align -Wshadow -Wstrict-aliasing=1 \ 67 | -Wswitch-enum -Wdeclaration-after-statement -Wstrict-prototypes \ 68 | -Wundef -Wpointer-arith -Wvla -Wformat=2 -Winit-self \ 69 | -Wfloat-equal -Wwrite-strings -Wredundant-decls -Wc++-compat \ 70 | -pedantic -fstack-protector-strong -fPIE -fPIC \ 71 | -fno-delete-null-pointer-checks -fwrapv -fno-strict-overflow 72 | 73 | DEBUGLEVEL ?=0 74 | 75 | DEBUGFLAGS += -DDEBUGLEVEL=$(DEBUGLEVEL) 76 | ifneq ($(DEBUGLEVEL), 0) 77 | QATFLAGS += -g -O0 78 | else 79 | QATFLAGS += -O3 80 | endif 81 | 82 | qatseqprod.o: qatseqprod.c 83 | $(CC) -c $(CFLAGS) $(QATFLAGS) $(DEBUGFLAGS) $^ -o $@ 84 | 85 | lib: qatseqprod.o 86 | $(AR) rc libqatseqprod.a $^ 87 | $(CC) -shared $^ $(LDFLAGS) -o libqatseqprod.so 88 | 89 | .PHONY: install 90 | install: lib 91 | [ -e $(DESTDIR)$(LIBDIR) ] || $(INSTALL) -d -m 755 $(DESTDIR)$(LIBDIR)/ 92 | [ -e $(DESTDIR)$(INCLUDEDIR) ] || $(INSTALL) -d -m 755 $(DESTDIR)$(INCLUDEDIR)/ 93 | $(INSTALL_PROGRAM) libqatseqprod.so $(DESTDIR)$(LIBDIR) 94 | $(INSTALL_DATA) libqatseqprod.a $(DESTDIR)$(LIBDIR) 95 | $(INSTALL_DATA) qatseqprod.h $(DESTDIR)$(INCLUDEDIR) 96 | @echo qatseqprod library successfully installed 97 | 98 | .PHONY: uninstall 99 | uninstall: 100 | $(RM) $(LIBDIR)/libqatseqprod.a 101 | $(RM) $(LIBDIR)/libqatseqprod.so 102 | $(RM) $(INCLUDEDIR)/qatseqprod.h 103 | @echo qatseqprod library successfully uninstalled 104 | 105 | clean: 106 | $(RM) *.o 107 | $(RM) libqatseqprod.a libqatseqprod.so 108 | -------------------------------------------------------------------------------- /src/qatseqprod.c: -------------------------------------------------------------------------------- 1 | /*************************************************************************** 2 | * 3 | * BSD LICENSE 4 | * 5 | * Copyright(c) 2007-2023 Intel Corporation. All rights reserved. 6 | * All rights reserved. 7 | * 8 | * Redistribution and use in source and binary forms, with or without 9 | * modification, are permitted provided that the following conditions 10 | * are met: 11 | * 12 | * * Redistributions of source code must retain the above copyright 13 | * notice, this list of conditions and the following disclaimer. 14 | * * Redistributions in binary form must reproduce the above copyright 15 | * notice, this list of conditions and the following disclaimer in 16 | * the documentation and/or other materials provided with the 17 | * distribution. 18 | * * Neither the name of Intel Corporation nor the names of its 19 | * contributors may be used to endorse or promote products derived 20 | * from this software without specific prior written permission. 21 | * 22 | * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 23 | * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 24 | * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 25 | * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 26 | * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 27 | * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 28 | * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 29 | * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 30 | * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 31 | * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 32 | * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 33 | * 34 | ***************************************************************************/ 35 | 36 | /** 37 | ***************************************************************************** 38 | * Dependencies 39 | *****************************************************************************/ 40 | #ifndef ZSTD_STATIC_LINKING_ONLY 41 | #define ZSTD_STATIC_LINKING_ONLY 42 | #endif 43 | #include "zstd.h" 44 | 45 | #include 46 | #include 47 | #include 48 | #include 49 | #include 50 | #include 51 | #include 52 | #include 53 | #include 54 | #include 55 | #include 56 | #include 57 | #include 58 | #include 59 | #include 60 | #include /* INT_MAX */ 61 | #include /* memset */ 62 | #include 63 | 64 | #ifdef INTREE 65 | #include "qat/cpa.h" 66 | #include "qat/cpa_dc.h" 67 | #include "qat/icp_sal_poll.h" 68 | #include "qat/icp_sal_user.h" 69 | #else 70 | #include "cpa.h" 71 | #include "cpa_dc.h" 72 | #include "icp_sal_poll.h" 73 | #include "icp_sal_user.h" 74 | #endif 75 | 76 | #include "qatseqprod.h" 77 | 78 | #ifdef INTREE 79 | #include "qat/qae_mem.h" 80 | #else 81 | #include "qae_mem.h" 82 | #endif 83 | 84 | #define KB (1024) 85 | 86 | #define COMP_LVL_MINIMUM (1) 87 | #define COMP_LVL_MAXIMUM (12) 88 | #define NUM_BLOCK_OF_RETRY_INTERVAL (1000) 89 | 90 | #define MAX_GRAB_RETRY (10) 91 | #define MAX_SEND_REQUEST_RETRY (5) 92 | #define MAX_DEVICES (256) 93 | 94 | #define SECTION_NAME_SIZE (32) 95 | 96 | #define INTER_SZ(src_sz) (2 * (src_sz)) 97 | #define COMPRESS_SRC_BUFF_SZ (ZSTD_BLOCKSIZE_MAX) 98 | 99 | #define ML_BITS 4 100 | #define ML_MASK ((1U << ML_BITS) - 1) 101 | #define RUN_BITS (8 - ML_BITS) 102 | #define RUN_MASK ((1U << RUN_BITS) - 1) 103 | 104 | #define LZ4MINMATCH 2 105 | 106 | /* Max latency of polling in the worst condition */ 107 | #define MAXTIMEOUT 2000000 108 | 109 | #define TIMESPENT(a, b) ((a.tv_sec * 1000000 + a.tv_usec) - (b.tv_sec * 1000000 + b.tv_usec)) 110 | 111 | /** QZSTD_Session_T: 112 | * This structure contains all session parameters including a buffer used to store 113 | * lz4s output for current session and other parameters 114 | */ 115 | typedef struct QZSTD_Session_S { 116 | int instHint; /*which instance we last used*/ 117 | unsigned char 118 | *qatIntermediateBuf; /* Buffer to store lz4s output for decoding */ 119 | unsigned char reqPhyContMem; /* 1: QAT requires physically contiguous memory */ 120 | CpaDcSessionSetupData 121 | sessionSetupData; /* Session set up data for this session */ 122 | unsigned int failOffloadCnt; /* Failed offloading requests counter */ 123 | } QZSTD_Session_T; 124 | 125 | /** QZSTD_Instance_T: 126 | * This structure contains instance parameter, every session need to grab one 127 | * instance to submit request 128 | */ 129 | typedef struct QZSTD_Instance_S { 130 | CpaInstanceInfo2 instanceInfo; 131 | CpaDcInstanceCapabilities instanceCap; 132 | CpaStatus jobStatus; 133 | CpaDcSessionSetupData sessionSetupData; 134 | CpaDcSessionHandle cpaSessHandle; 135 | CpaDcRqResults res; 136 | Cpa32U buffMetaSize; 137 | CpaStatus instStartStatus; 138 | unsigned char reqPhyContMem; /* 1: QAT requires physically contiguous memory */ 139 | 140 | /* Tracks memory where the intermediate buffers reside. */ 141 | CpaBufferList **intermediateBuffers; 142 | Cpa16U intermediateCnt; 143 | CpaBufferList *srcBuffer; 144 | CpaBufferList *destBuffer; 145 | 146 | unsigned int lock; 147 | unsigned char memSetup; 148 | unsigned char cpaSessSetup; 149 | unsigned char dcInstSetup; 150 | unsigned int numRetries; 151 | 152 | unsigned int seqNumIn; 153 | unsigned int seqNumOut; 154 | int cbStatus; 155 | } QZSTD_Instance_T; 156 | 157 | /** QZSTD_ProcessData_T: 158 | * Process data for controlling instance resource 159 | */ 160 | typedef struct QZSTD_ProcessData_S { 161 | int qzstdInitStatus; 162 | CpaInstanceHandle *dcInstHandle; 163 | QZSTD_Instance_T *qzstdInst; 164 | Cpa16U numInstances; 165 | pthread_mutex_t mutex; 166 | } QZSTD_ProcessData_T; 167 | 168 | typedef struct QZSTD_InstanceList_S { 169 | QZSTD_Instance_T instance; 170 | CpaInstanceHandle dcInstHandle; 171 | struct QZSTD_InstanceList_S *next; 172 | } QZSTD_InstanceList_T; 173 | 174 | typedef struct QZSTD_Hardware_S { 175 | QZSTD_InstanceList_T devices[MAX_DEVICES]; 176 | unsigned int devNum; 177 | unsigned int maxDevId; 178 | } QZSTD_Hardware_T; 179 | 180 | QZSTD_ProcessData_T gProcess = { 181 | .qzstdInitStatus = QZSTD_FAIL, 182 | .mutex = PTHREAD_MUTEX_INITIALIZER 183 | }; 184 | 185 | extern CpaStatus icp_adf_get_numDevices(Cpa32U *); 186 | 187 | int debugLevel = DEBUGLEVEL; 188 | 189 | #define QZSTD_DEBUG_PRINT(...) fprintf(stderr, __VA_ARGS__) 190 | 191 | /** 192 | * Print log message 193 | * @param l 194 | * Log level 195 | * 0: release mode, no debug log will be printed 196 | * 1: only print error info 197 | * 2: print events at every position 198 | * 3+: print all events and sequence data 199 | * @param ... 200 | * The format string 201 | */ 202 | #define QZSTD_LOG(l, ...){ \ 203 | if (l<=debugLevel) { \ 204 | QZSTD_DEBUG_PRINT(__FILE__ ": " __VA_ARGS__); \ 205 | } } 206 | 207 | const char *QZSTD_version(void) 208 | { 209 | return QZSTD_VERSION; 210 | } 211 | 212 | /** QZSTD_calloc: 213 | * This function is used to allocate contiguous or discontiguous memory(initialized to zero) 214 | * according to parameter and return pointer to allocated memory 215 | */ 216 | static void *QZSTD_calloc(size_t nb, size_t size, unsigned char reqPhyContMem) 217 | { 218 | if (!reqPhyContMem) { 219 | return calloc(nb, size); 220 | } else { 221 | return qaeMemAllocNUMA(nb * size, 0, 64); 222 | } 223 | } 224 | 225 | /** QZSTD_free: 226 | * This function needs to be called in pairs with QZSTD_calloc 227 | * to free memory by QZSTD_calloc. 228 | */ 229 | static void QZSTD_free(void *ptr, unsigned char reqPhyContMem) 230 | { 231 | if (!reqPhyContMem) { 232 | free(ptr); 233 | ptr = NULL; 234 | } else { 235 | qaeMemFreeNUMA(&ptr); 236 | ptr = NULL; 237 | } 238 | } 239 | 240 | /** QZSTD_virtToPhys: 241 | * Convert virtual address to physical 242 | */ 243 | static __inline CpaPhysicalAddr QZSTD_virtToPhys(void *virtAddr) 244 | { 245 | return (CpaPhysicalAddr)qaeVirtToPhysNUMA(virtAddr); 246 | } 247 | 248 | static QZSTD_InstanceList_T *QZSTD_getInstance(unsigned int devId, 249 | QZSTD_Hardware_T *qatHw) 250 | { 251 | QZSTD_InstanceList_T *instances; 252 | QZSTD_InstanceList_T *firstInstance; 253 | int i; 254 | 255 | if (devId >= MAX_DEVICES || NULL == qatHw) { 256 | return NULL; 257 | } 258 | 259 | instances = &qatHw->devices[devId]; 260 | firstInstance = instances->next; 261 | 262 | /* no instance */ 263 | if (NULL == firstInstance) { 264 | goto exit; 265 | } 266 | 267 | instances->next = firstInstance->next; 268 | 269 | /* last instance */ 270 | if (NULL == instances->next && qatHw->devNum > 0) { 271 | qatHw->devNum--; 272 | if (qatHw->maxDevId > 0 && devId == qatHw->maxDevId) { 273 | for (i = qatHw->maxDevId - 1; i >= 0; i--) { 274 | if (qatHw->devices[i].next) { 275 | qatHw->maxDevId = i; 276 | break; 277 | } 278 | } 279 | } 280 | } 281 | 282 | exit: 283 | return firstInstance; 284 | } 285 | 286 | static void QZSTD_clearDevices(QZSTD_Hardware_T *qatHw) 287 | { 288 | unsigned int i; 289 | if (NULL == qatHw || 0 == qatHw->devNum) { 290 | return; 291 | } 292 | 293 | for (i = 0; i <= qatHw->maxDevId; i++) { 294 | QZSTD_InstanceList_T *inst = QZSTD_getInstance(i, qatHw); 295 | while (inst) { 296 | free(inst); 297 | inst = NULL; 298 | inst = QZSTD_getInstance(i, qatHw); 299 | } 300 | } 301 | } 302 | 303 | /** QZSTD_stopQat: 304 | * Stop DC instance and QAT device 305 | */ 306 | static void QZSTD_stopQat(void) 307 | { 308 | int i; 309 | CpaStatus status = CPA_STATUS_SUCCESS; 310 | 311 | QZSTD_LOG(2, "Call stopQat\n"); 312 | if (NULL != gProcess.dcInstHandle && 313 | NULL != gProcess.qzstdInst) { 314 | for (i = 0; i < gProcess.numInstances; i++) { 315 | if (0 != gProcess.qzstdInst[i].dcInstSetup) { 316 | status = cpaDcStopInstance(gProcess.dcInstHandle[i]); 317 | if (CPA_STATUS_SUCCESS != status) { 318 | QZSTD_LOG(1, "Stop instance failed, status=%d\n", status); 319 | } 320 | } 321 | } 322 | 323 | free(gProcess.dcInstHandle); 324 | gProcess.dcInstHandle = NULL; 325 | free(gProcess.qzstdInst); 326 | gProcess.qzstdInst = NULL; 327 | } 328 | 329 | (void)icp_sal_userStop(); 330 | 331 | gProcess.numInstances = (Cpa16U)0; 332 | gProcess.qzstdInitStatus = QZSTD_FAIL; 333 | } 334 | 335 | static void QZSTD_removeSession(int i) 336 | { 337 | unsigned char reqPhyContMem = gProcess.qzstdInst[i].reqPhyContMem; 338 | int rc; 339 | 340 | if (0 == gProcess.qzstdInst[i].cpaSessSetup) { 341 | return; 342 | } 343 | 344 | /* Remove session */ 345 | if ((NULL != gProcess.dcInstHandle[i]) && 346 | (NULL != gProcess.qzstdInst[i].cpaSessHandle)) { 347 | /* polling here if there still are some responses haven't beed polled 348 | * if didn't poll there response, cpaDcRemoveSession will raise error message 349 | */ 350 | do { 351 | rc = icp_sal_DcPollInstance(gProcess.dcInstHandle[i], 0); 352 | } while (CPA_STATUS_SUCCESS == rc); 353 | cpaDcRemoveSession(gProcess.dcInstHandle[i], 354 | gProcess.qzstdInst[i].cpaSessHandle); 355 | QZSTD_free(gProcess.qzstdInst[i].cpaSessHandle, reqPhyContMem); 356 | gProcess.qzstdInst[i].cpaSessHandle = NULL; 357 | gProcess.qzstdInst[i].cpaSessSetup = 0; 358 | } 359 | } 360 | 361 | /** QZSTD_cleanUpInstMem: 362 | * Release the memory bound to corresponding instance 363 | */ 364 | static void QZSTD_cleanUpInstMem(int i) 365 | { 366 | int j; 367 | QZSTD_Instance_T *qzstdInst = &(gProcess.qzstdInst[i]); 368 | unsigned char reqPhyContMem = qzstdInst->reqPhyContMem; 369 | 370 | if (NULL != qzstdInst->intermediateBuffers) { 371 | for (j = 0; j < qzstdInst->intermediateCnt; j++) { 372 | if (NULL != qzstdInst->intermediateBuffers[j]) { 373 | if (NULL != qzstdInst->intermediateBuffers[j]->pBuffers) { 374 | if (NULL != qzstdInst->intermediateBuffers[j]->pBuffers->pData) { 375 | QZSTD_free(qzstdInst->intermediateBuffers[j]->pBuffers->pData, reqPhyContMem); 376 | qzstdInst->intermediateBuffers[j]->pBuffers->pData = NULL; 377 | } 378 | QZSTD_free(qzstdInst->intermediateBuffers[j]->pBuffers, reqPhyContMem); 379 | qzstdInst->intermediateBuffers[j]->pBuffers = NULL; 380 | } 381 | if (NULL != qzstdInst->intermediateBuffers[j]->pPrivateMetaData) { 382 | QZSTD_free(qzstdInst->intermediateBuffers[j]->pPrivateMetaData, reqPhyContMem); 383 | qzstdInst->intermediateBuffers[j]->pPrivateMetaData = NULL; 384 | } 385 | QZSTD_free(qzstdInst->intermediateBuffers[j], 0); 386 | qzstdInst->intermediateBuffers[j] = NULL; 387 | } 388 | } 389 | QZSTD_free(qzstdInst->intermediateBuffers, 0); 390 | qzstdInst->intermediateBuffers = NULL; 391 | } 392 | 393 | /*src buffer*/ 394 | if (NULL != qzstdInst->srcBuffer) { 395 | if (reqPhyContMem && qzstdInst->srcBuffer->pBuffers) { 396 | if (NULL != qzstdInst->srcBuffer->pBuffers->pData) { 397 | QZSTD_free(qzstdInst->srcBuffer->pBuffers->pData, reqPhyContMem); 398 | qzstdInst->srcBuffer->pBuffers->pData = NULL; 399 | } 400 | } 401 | if (NULL != qzstdInst->srcBuffer->pBuffers) { 402 | QZSTD_free(qzstdInst->srcBuffer->pBuffers, reqPhyContMem); 403 | qzstdInst->srcBuffer->pBuffers = NULL; 404 | } 405 | if (NULL != qzstdInst->srcBuffer->pPrivateMetaData) { 406 | QZSTD_free(qzstdInst->srcBuffer->pPrivateMetaData, reqPhyContMem); 407 | qzstdInst->srcBuffer->pPrivateMetaData = NULL; 408 | } 409 | QZSTD_free(qzstdInst->srcBuffer, 0); 410 | qzstdInst->srcBuffer = NULL; 411 | } 412 | 413 | /*dest buffer*/ 414 | if (NULL != qzstdInst->destBuffer) { 415 | if (NULL != qzstdInst->destBuffer->pBuffers) { 416 | QZSTD_free(qzstdInst->destBuffer->pBuffers, reqPhyContMem); 417 | qzstdInst->destBuffer->pBuffers = NULL; 418 | } 419 | if (NULL != qzstdInst->destBuffer->pPrivateMetaData) { 420 | QZSTD_free(qzstdInst->destBuffer->pPrivateMetaData, reqPhyContMem); 421 | qzstdInst->destBuffer->pPrivateMetaData = NULL; 422 | } 423 | QZSTD_free(qzstdInst->destBuffer, 0); 424 | qzstdInst->destBuffer = NULL; 425 | } 426 | } 427 | 428 | void QZSTD_stopQatDevice(void) 429 | { 430 | pthread_mutex_lock(&gProcess.mutex); 431 | if (QZSTD_OK == gProcess.qzstdInitStatus) { 432 | int i = 0; 433 | 434 | for (i = 0; i < gProcess.numInstances; i++) { 435 | if (0 != gProcess.qzstdInst[i].cpaSessSetup) { 436 | QZSTD_removeSession(i); 437 | } 438 | if (0 != gProcess.qzstdInst[i].memSetup) { 439 | QZSTD_cleanUpInstMem(i); 440 | } 441 | } 442 | QZSTD_stopQat(); 443 | } 444 | if (QZSTD_STARTED == gProcess.qzstdInitStatus) { 445 | (void)icp_sal_userStop(); 446 | gProcess.qzstdInitStatus = QZSTD_FAIL; 447 | } 448 | pthread_mutex_unlock(&gProcess.mutex); 449 | } 450 | 451 | static int QZSTD_setInstance(unsigned int devId, 452 | QZSTD_InstanceList_T *newInstance, 453 | QZSTD_Hardware_T *qatHw) 454 | { 455 | QZSTD_InstanceList_T *instances; 456 | 457 | if (devId >= MAX_DEVICES || NULL == newInstance || NULL == qatHw || 458 | NULL != newInstance->next) { 459 | return QZSTD_FAIL; 460 | } 461 | 462 | instances = &qatHw->devices[devId]; 463 | 464 | /* first instance */ 465 | if (NULL == instances->next) { 466 | qatHw->devNum++; 467 | } 468 | 469 | while (instances->next) { 470 | instances = instances->next; 471 | } 472 | instances->next = newInstance; 473 | 474 | if (devId > qatHw->maxDevId) { 475 | qatHw->maxDevId = devId; 476 | } 477 | 478 | return QZSTD_OK; 479 | } 480 | 481 | const char *QZSTD_getSectionName(void) 482 | { 483 | static char sectionName[SECTION_NAME_SIZE]; 484 | int len; 485 | char *preSectionName; 486 | preSectionName = getenv("QAT_SECTION_NAME"); 487 | 488 | if (!preSectionName || !(len = strlen(preSectionName))) { 489 | preSectionName = (char *)"SHIM"; 490 | } else if (len >= SECTION_NAME_SIZE) { 491 | QZSTD_LOG(1, "The length of QAT_SECTION_NAME exceeds the limit\n"); 492 | } 493 | strncpy(sectionName, preSectionName, SECTION_NAME_SIZE - 1); 494 | sectionName[SECTION_NAME_SIZE - 1] = '\0'; 495 | return sectionName; 496 | } 497 | 498 | static int QZSTD_salUserStart(void) 499 | { 500 | #ifndef INTREE 501 | Cpa32U pcieCount; 502 | 503 | if (CPA_STATUS_SUCCESS != icp_adf_get_numDevices(&pcieCount)) { 504 | QZSTD_LOG(1, "icp_adf_get_numDevices failed\n"); 505 | return QZSTD_FAIL; 506 | } 507 | 508 | if (0 == pcieCount) { 509 | QZSTD_LOG(1, 510 | "There is no QAT device available, please check QAT device status\n"); 511 | return QZSTD_FAIL; 512 | } 513 | #else 514 | if (CPA_TRUE != icp_sal_userIsQatAvailable()) { 515 | QZSTD_LOG(1, 516 | "There is no QAT device available, please check QAT device status\n"); 517 | return QZSTD_FAIL; 518 | } 519 | #endif 520 | 521 | if (CPA_STATUS_SUCCESS != icp_sal_userStart(QZSTD_getSectionName())) { 522 | QZSTD_LOG(1, "icp_sal_userStart failed\n"); 523 | return QZSTD_FAIL; 524 | } 525 | 526 | return QZSTD_OK; 527 | } 528 | 529 | static int QZSTD_getAndShuffleInstance(void) 530 | { 531 | int i; 532 | unsigned int devId = 0; 533 | QZSTD_Hardware_T *qatHw = NULL; 534 | unsigned int instanceFound = 0; 535 | unsigned int instanceMatched = 0; 536 | QZSTD_InstanceList_T *newInst; 537 | if (CPA_STATUS_SUCCESS != cpaDcGetNumInstances(&gProcess.numInstances)) { 538 | QZSTD_LOG(1, "cpaDcGetNumInstances failed\n"); 539 | goto exit; 540 | } 541 | 542 | gProcess.dcInstHandle = (CpaInstanceHandle *)calloc( 543 | gProcess.numInstances, sizeof(CpaInstanceHandle)); 544 | gProcess.qzstdInst = (QZSTD_Instance_T *)calloc(gProcess.numInstances, 545 | sizeof(QZSTD_Instance_T)); 546 | if (NULL == gProcess.dcInstHandle || NULL == gProcess.qzstdInst) { 547 | QZSTD_LOG(1, "calloc for qzstdInst failed\n"); 548 | goto exit; 549 | } 550 | 551 | if (CPA_STATUS_SUCCESS != cpaDcGetInstances( 552 | gProcess.numInstances, gProcess.dcInstHandle)) { 553 | QZSTD_LOG(1, "cpaDcGetInstances failed\n"); 554 | goto exit; 555 | } 556 | 557 | qatHw = (QZSTD_Hardware_T *)calloc(1, sizeof(QZSTD_Hardware_T)); 558 | if (NULL == qatHw) { 559 | QZSTD_LOG(1, "calloc for qatHw failed\n"); 560 | goto exit; 561 | } 562 | for (i = 0; i < gProcess.numInstances; i++) { 563 | newInst = (QZSTD_InstanceList_T *)calloc(1, sizeof(QZSTD_InstanceList_T)); 564 | if (NULL == newInst) { 565 | QZSTD_LOG(1, "calloc failed\n"); 566 | goto exit; 567 | } 568 | 569 | if (CPA_STATUS_SUCCESS != cpaDcInstanceGetInfo2( 570 | gProcess.dcInstHandle[i], &newInst->instance.instanceInfo)) { 571 | QZSTD_LOG(1, "cpaDcInstanceGetInfo2 failed\n"); 572 | free(newInst); 573 | newInst = NULL; 574 | goto exit; 575 | } 576 | 577 | if (CPA_STATUS_SUCCESS != cpaDcQueryCapabilities( 578 | gProcess.dcInstHandle[i], &newInst->instance.instanceCap)) { 579 | QZSTD_LOG(1, "cpaDcQueryCapabilities failed\n"); 580 | free(newInst); 581 | newInst = NULL; 582 | goto exit; 583 | } 584 | 585 | newInst->instance.lock = 0; 586 | newInst->instance.memSetup = 0; 587 | newInst->instance.cpaSessSetup = 0; 588 | newInst->instance.dcInstSetup = 0; 589 | newInst->instance.numRetries = 0; 590 | newInst->dcInstHandle = gProcess.dcInstHandle[i]; 591 | 592 | devId = newInst->instance.instanceInfo.physInstId.packageId; 593 | if (QZSTD_OK != QZSTD_setInstance(devId, newInst, qatHw)) { 594 | QZSTD_LOG(1, "QZSTD_setInstance on device %d failed\n", devId); 595 | free(newInst); 596 | newInst = NULL; 597 | goto exit; 598 | } 599 | } 600 | 601 | /* shuffle instance */ 602 | for (i = 0; instanceFound < gProcess.numInstances; i++) { 603 | devId = i % (qatHw->maxDevId + 1); 604 | newInst = QZSTD_getInstance(devId, qatHw); 605 | if (NULL == newInst) { 606 | continue; 607 | } 608 | instanceFound++; 609 | 610 | /* check lz4s support */ 611 | if (!newInst->instance.instanceCap.checksumXXHash32 || 612 | !newInst->instance.instanceCap.statelessLZ4SCompression) { 613 | free(newInst); 614 | newInst = NULL; 615 | continue; 616 | } 617 | 618 | if (newInst->instance.instanceInfo.requiresPhysicallyContiguousMemory) { 619 | newInst->instance.reqPhyContMem = 1; 620 | } else { 621 | newInst->instance.reqPhyContMem = 0; 622 | } 623 | 624 | memcpy(&gProcess.qzstdInst[instanceMatched], &newInst->instance, 625 | sizeof(QZSTD_Instance_T)); 626 | gProcess.dcInstHandle[instanceMatched] = newInst->dcInstHandle; 627 | free(newInst); 628 | newInst = NULL; 629 | instanceMatched++; 630 | } 631 | 632 | if (0 == instanceMatched) { 633 | QZSTD_LOG(1, "No instance with matching capabilities\n"); 634 | goto exit; 635 | } 636 | 637 | QZSTD_clearDevices(qatHw); 638 | free(qatHw); 639 | qatHw = NULL; 640 | 641 | return QZSTD_OK; 642 | 643 | exit: 644 | if (qatHw) { 645 | QZSTD_clearDevices(qatHw); 646 | free(qatHw); 647 | qatHw = NULL; 648 | } 649 | if (NULL != gProcess.dcInstHandle) { 650 | free(gProcess.dcInstHandle); 651 | gProcess.dcInstHandle = NULL; 652 | } 653 | if (NULL != gProcess.qzstdInst) { 654 | free(gProcess.qzstdInst); 655 | gProcess.qzstdInst = NULL; 656 | } 657 | 658 | if (instanceFound && 0 == instanceMatched) { 659 | return QZSTD_UNSUPPORTED; 660 | } else { 661 | return QZSTD_FAIL; 662 | } 663 | } 664 | 665 | static void QZSTD_dcCallback(void *cbDataTag, CpaStatus stat) 666 | { 667 | if (NULL != cbDataTag) { 668 | QZSTD_Instance_T *qzstdInst = (QZSTD_Instance_T *)cbDataTag; 669 | qzstdInst->seqNumOut++; 670 | if (qzstdInst->seqNumIn != qzstdInst->seqNumOut) { 671 | return; 672 | } 673 | 674 | if (CPA_DC_OK == stat) { 675 | qzstdInst->cbStatus = QZSTD_OK; 676 | } else { 677 | qzstdInst->cbStatus = QZSTD_FAIL; 678 | } 679 | } 680 | } 681 | 682 | /** QZSTD_allocInstMem: 683 | * Allocate memory for corresponding instance 684 | */ 685 | static int QZSTD_allocInstMem(int i) 686 | { 687 | int j; 688 | CpaStatus status; 689 | CpaStatus rc; 690 | unsigned int srcSz; 691 | unsigned int interSz; 692 | unsigned char reqPhyContMem = gProcess.qzstdInst[i].reqPhyContMem; 693 | 694 | rc = QZSTD_OK; 695 | srcSz = COMPRESS_SRC_BUFF_SZ; 696 | interSz = INTER_SZ(srcSz); 697 | 698 | status = 699 | cpaDcBufferListGetMetaSize(gProcess.dcInstHandle[i], 1, 700 | &(gProcess.qzstdInst[i].buffMetaSize)); 701 | if (CPA_STATUS_SUCCESS != status) { 702 | QZSTD_LOG(1, "cpaDcBufferListGetMetaSize failed\n"); 703 | goto cleanup; 704 | } 705 | 706 | status = cpaDcGetNumIntermediateBuffers( 707 | gProcess.dcInstHandle[i], &(gProcess.qzstdInst[i].intermediateCnt)); 708 | if (CPA_STATUS_SUCCESS != status) { 709 | QZSTD_LOG(1, "cpaDcGetNumIntermediateBuffers failed\n"); 710 | goto cleanup; 711 | } 712 | gProcess.qzstdInst[i].intermediateBuffers = 713 | (CpaBufferList **)QZSTD_calloc((size_t)gProcess.qzstdInst[i].intermediateCnt, 714 | sizeof(CpaBufferList *), 0); 715 | if (NULL == gProcess.qzstdInst[i].intermediateBuffers) { 716 | QZSTD_LOG(1, "Failed to allocate memory\n"); 717 | goto cleanup; 718 | } 719 | 720 | for (j = 0; j < gProcess.qzstdInst[i].intermediateCnt; j++) { 721 | gProcess.qzstdInst[i].intermediateBuffers[j] = 722 | (CpaBufferList *)QZSTD_calloc(1, sizeof(CpaBufferList), 0); 723 | if (NULL == gProcess.qzstdInst[i].intermediateBuffers[j]) { 724 | QZSTD_LOG(1, "Failed to allocate memory\n"); 725 | goto cleanup; 726 | } 727 | if (0 != gProcess.qzstdInst[i].buffMetaSize) { 728 | gProcess.qzstdInst[i].intermediateBuffers[j]->pPrivateMetaData = 729 | QZSTD_calloc(1, (size_t)(gProcess.qzstdInst[i].buffMetaSize), reqPhyContMem); 730 | if (NULL == 731 | gProcess.qzstdInst[i].intermediateBuffers[j]->pPrivateMetaData) { 732 | QZSTD_LOG(1, "Failed to allocate memory\n"); 733 | goto cleanup; 734 | } 735 | } 736 | 737 | gProcess.qzstdInst[i].intermediateBuffers[j]->pBuffers = 738 | (CpaFlatBuffer *)QZSTD_calloc(1, sizeof(CpaFlatBuffer), reqPhyContMem); 739 | if (NULL == gProcess.qzstdInst[i].intermediateBuffers[j]->pBuffers) { 740 | QZSTD_LOG(1, "Failed to allocate memory\n"); 741 | goto cleanup; 742 | } 743 | 744 | gProcess.qzstdInst[i].intermediateBuffers[j]->pBuffers->pData = 745 | (Cpa8U *)QZSTD_calloc(1, interSz, reqPhyContMem); 746 | if (NULL == 747 | gProcess.qzstdInst[i].intermediateBuffers[j]->pBuffers->pData) { 748 | QZSTD_LOG(1, "Failed to allocate memory\n"); 749 | goto cleanup; 750 | } 751 | 752 | gProcess.qzstdInst[i].intermediateBuffers[j]->pBuffers->dataLenInBytes = 753 | interSz; 754 | } 755 | gProcess.qzstdInst[i].srcBuffer = 756 | (CpaBufferList *)QZSTD_calloc(1, sizeof(CpaBufferList), 0); 757 | if (NULL == gProcess.qzstdInst[i].srcBuffer) { 758 | QZSTD_LOG(1, "Failed to allocate memory\n"); 759 | goto cleanup; 760 | } 761 | gProcess.qzstdInst[i].srcBuffer->numBuffers = 1; 762 | 763 | if (0 != gProcess.qzstdInst[i].buffMetaSize) { 764 | gProcess.qzstdInst[i].srcBuffer->pPrivateMetaData = 765 | QZSTD_calloc(1, (size_t)gProcess.qzstdInst[i].buffMetaSize, reqPhyContMem); 766 | if (NULL == gProcess.qzstdInst[i].srcBuffer->pPrivateMetaData) { 767 | QZSTD_LOG(1, "Failed to allocate memory\n"); 768 | goto cleanup; 769 | } 770 | } 771 | 772 | gProcess.qzstdInst[i].srcBuffer->pBuffers = 773 | (CpaFlatBuffer *)QZSTD_calloc(1, sizeof(CpaFlatBuffer), reqPhyContMem); 774 | if (NULL == gProcess.qzstdInst[i].srcBuffer->pBuffers) { 775 | QZSTD_LOG(1, "Failed to allocate memory\n"); 776 | goto cleanup; 777 | } 778 | if (reqPhyContMem) { 779 | gProcess.qzstdInst[i].srcBuffer->pBuffers->pData = 780 | (Cpa8U *)QZSTD_calloc(1, srcSz, reqPhyContMem); 781 | if (NULL == gProcess.qzstdInst[i].srcBuffer->pBuffers->pData) { 782 | QZSTD_LOG(1, "Failed to allocate memory\n"); 783 | goto cleanup; 784 | } 785 | } else { 786 | gProcess.qzstdInst[i].srcBuffer->pBuffers->pData = NULL; 787 | } 788 | 789 | gProcess.qzstdInst[i].destBuffer = 790 | (CpaBufferList *)QZSTD_calloc(1, sizeof(CpaBufferList), 0); 791 | if (NULL == gProcess.qzstdInst[i].destBuffer) { 792 | QZSTD_LOG(1, "Failed to allocate memory\n"); 793 | goto cleanup; 794 | } 795 | gProcess.qzstdInst[i].destBuffer->numBuffers = 1; 796 | 797 | if (0 != gProcess.qzstdInst[i].buffMetaSize) { 798 | gProcess.qzstdInst[i].destBuffer->pPrivateMetaData = 799 | QZSTD_calloc(1, (size_t)gProcess.qzstdInst[i].buffMetaSize, reqPhyContMem); 800 | if (NULL == gProcess.qzstdInst[i].destBuffer->pPrivateMetaData) { 801 | QZSTD_LOG(1, "Failed to allocate memory\n"); 802 | goto cleanup; 803 | } 804 | } 805 | 806 | gProcess.qzstdInst[i].destBuffer->pBuffers = 807 | (CpaFlatBuffer *)QZSTD_calloc(1, sizeof(CpaFlatBuffer), reqPhyContMem); 808 | if (NULL == gProcess.qzstdInst[i].destBuffer->pBuffers) { 809 | QZSTD_LOG(1, "Failed to allocate memory\n"); 810 | goto cleanup; 811 | } 812 | 813 | gProcess.qzstdInst[i].memSetup = 1; 814 | 815 | done_inst: 816 | return rc; 817 | 818 | cleanup: 819 | QZSTD_cleanUpInstMem(i); 820 | rc = QZSTD_FAIL; 821 | goto done_inst; 822 | } 823 | 824 | static int QZSTD_startDcInstance(int i) 825 | { 826 | int rc = QZSTD_OK; 827 | 828 | if (CPA_STATUS_SUCCESS != cpaDcSetAddressTranslation( 829 | gProcess.dcInstHandle[i], (CpaVirtualToPhysical)QZSTD_virtToPhys)) { 830 | QZSTD_LOG(1, "cpaDcSetAddressTranslation failed\n"); 831 | rc = QZSTD_FAIL; 832 | goto done; 833 | } 834 | 835 | gProcess.qzstdInst[i].instStartStatus = cpaDcStartInstance( 836 | gProcess.dcInstHandle[i], gProcess.qzstdInst[i].intermediateCnt, 837 | gProcess.qzstdInst[i].intermediateBuffers); 838 | if (CPA_STATUS_SUCCESS != gProcess.qzstdInst[i].instStartStatus) { 839 | QZSTD_LOG(1, "cpaDcStartInstance failed\n"); 840 | rc = QZSTD_FAIL; 841 | goto done; 842 | } 843 | 844 | gProcess.qzstdInst[i].seqNumIn = 0; 845 | gProcess.qzstdInst[i].seqNumOut = 0; 846 | gProcess.qzstdInst[i].dcInstSetup = 1; 847 | 848 | done: 849 | return rc; 850 | } 851 | 852 | static int QZSTD_cpaInitSess(QZSTD_Session_T *sess, int i) 853 | { 854 | Cpa32U sessionSize = 0; 855 | Cpa32U ctxSize = 0; 856 | unsigned char reqPhyContMem = gProcess.qzstdInst[i].reqPhyContMem; 857 | 858 | /*setup and start DC session*/ 859 | if (CPA_STATUS_SUCCESS != cpaDcGetSessionSize(gProcess.dcInstHandle[i], 860 | &sess->sessionSetupData, &sessionSize, &ctxSize)) { 861 | QZSTD_LOG(1, "cpaDcGetSessionSize failed\n"); 862 | return QZSTD_FAIL; 863 | } 864 | 865 | gProcess.qzstdInst[i].cpaSessHandle = QZSTD_calloc(1, (size_t)(sessionSize), 866 | reqPhyContMem); 867 | if (NULL == gProcess.qzstdInst[i].cpaSessHandle) { 868 | QZSTD_LOG(1, "Failed to allocate memory\n"); 869 | return QZSTD_FAIL; 870 | } 871 | 872 | if (CPA_STATUS_SUCCESS != cpaDcInitSession( 873 | gProcess.dcInstHandle[i], gProcess.qzstdInst[i].cpaSessHandle, 874 | &sess->sessionSetupData, NULL, QZSTD_dcCallback)) { 875 | QZSTD_LOG(1, "cpaDcInitSession failed\n"); 876 | QZSTD_free(gProcess.qzstdInst[i].cpaSessHandle, reqPhyContMem); 877 | gProcess.qzstdInst[i].cpaSessHandle = NULL; 878 | return QZSTD_FAIL; 879 | } 880 | 881 | gProcess.qzstdInst[i].sessionSetupData = sess->sessionSetupData; 882 | gProcess.qzstdInst[i].cpaSessSetup = 1; 883 | 884 | return QZSTD_OK; 885 | } 886 | 887 | static int QZSTD_cpaUpdateSess(QZSTD_Session_T *sess, int i) 888 | { 889 | unsigned char reqPhyContMem = gProcess.qzstdInst[i].reqPhyContMem; 890 | gProcess.qzstdInst[i].cpaSessSetup = 0; 891 | 892 | /* Remove session */ 893 | if (CPA_STATUS_SUCCESS != cpaDcRemoveSession( 894 | gProcess.dcInstHandle[i], gProcess.qzstdInst[i].cpaSessHandle)) { 895 | QZSTD_LOG(1, "cpaDcRemoveSession failed\n"); 896 | return QZSTD_FAIL; 897 | } 898 | 899 | QZSTD_free(gProcess.qzstdInst[i].cpaSessHandle, reqPhyContMem); 900 | gProcess.qzstdInst[i].cpaSessHandle = NULL; 901 | 902 | return QZSTD_cpaInitSess(sess, i); 903 | } 904 | 905 | static int QZSTD_grabInstance(int hint) 906 | { 907 | int i, j, rc, f; 908 | 909 | if (hint >= gProcess.numInstances || hint < 0) { 910 | hint = 0; 911 | } 912 | 913 | /*otherwise loop through all of them*/ 914 | f = 0; 915 | for (j = 0; j < MAX_GRAB_RETRY; j++) { 916 | for (i = 0; i < gProcess.numInstances; i++) { 917 | if (f == 0) { 918 | i = hint; 919 | f = 1; 920 | }; 921 | rc = __sync_lock_test_and_set(&(gProcess.qzstdInst[i].lock), 1); 922 | if (0 == rc) { 923 | return i; 924 | } 925 | } 926 | } 927 | return -1; 928 | } 929 | 930 | static void QZSTD_releaseInstance(int i) 931 | { 932 | __sync_lock_release(&(gProcess.qzstdInst[i].lock)); 933 | } 934 | 935 | static void QZSTD_setupSess(QZSTD_Session_T *zstdSess) 936 | { 937 | zstdSess->instHint = -1; 938 | zstdSess->sessionSetupData.compType = CPA_DC_LZ4S; 939 | zstdSess->sessionSetupData.autoSelectBestHuffmanTree = CPA_DC_ASB_ENABLED; 940 | zstdSess->sessionSetupData.sessDirection = CPA_DC_DIR_COMPRESS; 941 | zstdSess->sessionSetupData.sessState = CPA_DC_STATELESS; 942 | zstdSess->sessionSetupData.checksum = CPA_DC_XXHASH32; 943 | zstdSess->sessionSetupData.huffType = CPA_DC_HT_STATIC; 944 | zstdSess->sessionSetupData.minMatch = CPA_DC_MIN_3_BYTE_MATCH; 945 | zstdSess->failOffloadCnt = 0; 946 | } 947 | 948 | int QZSTD_startQatDevice(void) 949 | { 950 | pthread_mutex_lock(&gProcess.mutex); 951 | 952 | if (QZSTD_FAIL == gProcess.qzstdInitStatus) { 953 | gProcess.qzstdInitStatus = QZSTD_OK == QZSTD_salUserStart() ? QZSTD_STARTED : 954 | QZSTD_FAIL; 955 | } 956 | 957 | if (QZSTD_STARTED == gProcess.qzstdInitStatus) { 958 | gProcess.qzstdInitStatus = QZSTD_OK == QZSTD_getAndShuffleInstance() ? 959 | QZSTD_OK : QZSTD_STARTED; 960 | } 961 | QZSTD_LOG(2, "InitStatus: %d\n", gProcess.qzstdInitStatus); 962 | pthread_mutex_unlock(&gProcess.mutex); 963 | return gProcess.qzstdInitStatus; 964 | } 965 | 966 | static unsigned isLittleEndian(void) 967 | { 968 | const union { 969 | unsigned int u; 970 | unsigned char c[4]; 971 | } one = {1}; /* don't use static : performance detrimental */ 972 | return one.c[0]; 973 | } 974 | 975 | static unsigned short read16(const void *memPtr) 976 | { 977 | unsigned short val; 978 | memcpy(&val, memPtr, sizeof(val)); 979 | return val; 980 | } 981 | 982 | static unsigned short readLE16(const void *memPtr) 983 | { 984 | if (isLittleEndian()) { 985 | return read16(memPtr); 986 | } else { 987 | const unsigned char *p = (const unsigned char *)memPtr; 988 | return (unsigned short)((unsigned short)p[0] + (p[1] << 8)); 989 | } 990 | } 991 | 992 | void *QZSTD_createSeqProdState(void) 993 | { 994 | QZSTD_Session_T *zstdSess = (QZSTD_Session_T *)calloc(1, 995 | sizeof(QZSTD_Session_T)); 996 | QZSTD_setupSess(zstdSess); 997 | return (void *)zstdSess; 998 | } 999 | 1000 | void QZSTD_freeSeqProdState(void *sequenceProducerState) 1001 | { 1002 | QZSTD_Session_T *zstdSess = (QZSTD_Session_T *)sequenceProducerState; 1003 | if (zstdSess) { 1004 | if (zstdSess->qatIntermediateBuf) { 1005 | QZSTD_free(zstdSess->qatIntermediateBuf, zstdSess->reqPhyContMem); 1006 | zstdSess->qatIntermediateBuf = NULL; 1007 | } 1008 | free(zstdSess); 1009 | zstdSess = NULL; 1010 | } 1011 | } 1012 | 1013 | static size_t QZSTD_decLz4s(ZSTD_Sequence *outSeqs, size_t outSeqsCapacity, 1014 | unsigned char *lz4sBuff, unsigned int lz4sBufSize) 1015 | { 1016 | unsigned char *ip = lz4sBuff; 1017 | unsigned char *endip = lz4sBuff + lz4sBufSize; 1018 | unsigned int histLiteralLen = 0; 1019 | 1020 | size_t seqsIdx = 0; 1021 | 1022 | while (ip < endip && lz4sBufSize > 0) { 1023 | size_t length = 0; 1024 | size_t offset = 0; 1025 | size_t literalLen = 0, matchlen = 0; 1026 | /* get literal length */ 1027 | unsigned const token = *ip++; 1028 | if ((length = (token >> ML_BITS)) == RUN_MASK) { 1029 | unsigned s; 1030 | do { 1031 | s = *ip++; 1032 | length += s; 1033 | } while (s == 255); 1034 | } 1035 | literalLen = length; 1036 | ip += length; 1037 | if (ip == endip) { /* Meet the end of the LZ4 sequence */ 1038 | literalLen += histLiteralLen; 1039 | outSeqs[seqsIdx].litLength = literalLen; 1040 | outSeqs[seqsIdx].offset = offset; 1041 | outSeqs[seqsIdx].matchLength = matchlen; 1042 | QZSTD_LOG(3, "Last sequence, literalLen: %lu, offset: %lu, matchlen: %lu\n", 1043 | literalLen, offset, matchlen); 1044 | break; 1045 | } 1046 | 1047 | /* get matchPos */ 1048 | offset = readLE16(ip); 1049 | ip += 2; 1050 | 1051 | /* get match length */ 1052 | length = token & ML_MASK; 1053 | if (length == ML_MASK) { 1054 | unsigned s; 1055 | do { 1056 | s = *ip++; 1057 | length += s; 1058 | } while (s == 255); 1059 | } 1060 | if (length != 0) { 1061 | length += LZ4MINMATCH; 1062 | matchlen = (unsigned short)length; 1063 | literalLen += histLiteralLen; 1064 | 1065 | /* update ZSTD_Sequence */ 1066 | outSeqs[seqsIdx].offset = offset; 1067 | outSeqs[seqsIdx].litLength = literalLen; 1068 | outSeqs[seqsIdx].matchLength = matchlen; 1069 | QZSTD_LOG(3, "sequence, literalLen: %lu, offset: %lu, matchlen: %lu\n", 1070 | literalLen, offset, matchlen); 1071 | histLiteralLen = 0; 1072 | ++seqsIdx; 1073 | if (seqsIdx >= (outSeqsCapacity - 1)) { 1074 | QZSTD_LOG(1, "Sequence exceed capacity\n"); 1075 | return ZSTD_SEQUENCE_PRODUCER_ERROR; 1076 | } 1077 | } else { 1078 | if (literalLen > 0) { 1079 | /* When match length is 0, the literalLen needs to be 1080 | temporarily stored and processed together with the next data 1081 | block.*/ 1082 | histLiteralLen += literalLen; 1083 | } 1084 | } 1085 | } 1086 | if (ip != endip) { 1087 | QZSTD_LOG(1, "Unexpected decode error\n"); 1088 | return ZSTD_SEQUENCE_PRODUCER_ERROR; 1089 | } 1090 | return ++seqsIdx; 1091 | } 1092 | 1093 | static inline void QZSTD_castConstPointer(unsigned char **dest, 1094 | const void **src) 1095 | { 1096 | memcpy(dest, src, sizeof(char *)); 1097 | } 1098 | 1099 | static inline int QZSTD_isTimeOut(struct timeval timeStart, 1100 | struct timeval timeNow) 1101 | { 1102 | long long timeSpent = TIMESPENT(timeNow, timeStart); 1103 | return timeSpent > MAXTIMEOUT ? 1 : 0; 1104 | } 1105 | 1106 | size_t qatSequenceProducer( 1107 | void *sequenceProducerState, ZSTD_Sequence *outSeqs, size_t outSeqsCapacity, 1108 | const void *src, size_t srcSize, 1109 | const void *dict, size_t dictSize, 1110 | int compressionLevel, 1111 | size_t windowSize) 1112 | { 1113 | int i; 1114 | size_t rc = ZSTD_SEQUENCE_PRODUCER_ERROR; 1115 | int qrc = CPA_STATUS_FAIL; 1116 | CpaDcOpData opData; 1117 | int retry_cnt = MAX_SEND_REQUEST_RETRY; 1118 | struct timeval timeStart; 1119 | struct timeval timeNow; 1120 | QZSTD_Session_T *zstdSess = (QZSTD_Session_T *)sequenceProducerState; 1121 | Cpa32U intermediateBufLen = 0; 1122 | 1123 | if (windowSize < (srcSize < 32 * KB ? srcSize : 32 * KB) || dictSize > 0 || 1124 | dict) { 1125 | QZSTD_LOG(2, 1126 | "Currently not use windowsSize and not support dictionary, windowsSize: %lu, srcSize: %lu, dictSize: %lu\n", 1127 | windowSize, srcSize, dictSize); 1128 | return ZSTD_SEQUENCE_PRODUCER_ERROR; 1129 | } 1130 | 1131 | /* QAT only support L1-L12 */ 1132 | if (compressionLevel < COMP_LVL_MINIMUM || 1133 | compressionLevel > COMP_LVL_MAXIMUM) { 1134 | QZSTD_LOG(1, "Only can offload L1-L12 to QAT, current compression level: %d\n" 1135 | , compressionLevel); 1136 | return ZSTD_SEQUENCE_PRODUCER_ERROR; 1137 | } 1138 | 1139 | /* check hardware initialization status */ 1140 | if (gProcess.qzstdInitStatus != QZSTD_OK) { 1141 | zstdSess->failOffloadCnt++; 1142 | if (zstdSess->failOffloadCnt >= NUM_BLOCK_OF_RETRY_INTERVAL) { 1143 | zstdSess->failOffloadCnt = 0; 1144 | if (QZSTD_startQatDevice() != QZSTD_OK) { 1145 | QZSTD_LOG(1, "Tried to restart QAT device, but failed\n"); 1146 | return ZSTD_SEQUENCE_PRODUCER_ERROR; 1147 | } 1148 | } else { 1149 | QZSTD_LOG(1, "The hardware was not successfully started\n"); 1150 | return ZSTD_SEQUENCE_PRODUCER_ERROR; 1151 | } 1152 | } 1153 | 1154 | zstdSess->sessionSetupData.compLevel = (CpaDcCompLvl)compressionLevel; 1155 | 1156 | i = QZSTD_grabInstance(zstdSess->instHint); 1157 | if (-1 == i) { 1158 | QZSTD_LOG(1, "Failed to grab instance\n"); 1159 | return ZSTD_SEQUENCE_PRODUCER_ERROR; 1160 | } 1161 | 1162 | zstdSess->instHint = i; 1163 | zstdSess->reqPhyContMem = gProcess.qzstdInst[i].reqPhyContMem; 1164 | 1165 | /* allocate instance's buffer */ 1166 | if (0 == gProcess.qzstdInst[i].memSetup) { 1167 | if (QZSTD_OK != QZSTD_allocInstMem(i)) { 1168 | QZSTD_LOG(1, "Failed to allocate instance related memory\n"); 1169 | rc = ZSTD_SEQUENCE_PRODUCER_ERROR; 1170 | goto exit; 1171 | } 1172 | } 1173 | 1174 | /* start Dc Instance */ 1175 | if (0 == gProcess.qzstdInst[i].dcInstSetup) { 1176 | if (QZSTD_OK != QZSTD_startDcInstance(i)) { 1177 | QZSTD_LOG(1, "Failed to start DC instance\n"); 1178 | rc = ZSTD_SEQUENCE_PRODUCER_ERROR; 1179 | goto exit; 1180 | } 1181 | } 1182 | 1183 | /* init cpaSessHandle */ 1184 | if (0 == gProcess.qzstdInst[i].cpaSessSetup) { 1185 | if (QZSTD_OK != QZSTD_cpaInitSess(zstdSess, i)) { 1186 | QZSTD_LOG(1, "Failed to init sess\n"); 1187 | rc = ZSTD_SEQUENCE_PRODUCER_ERROR; 1188 | goto exit; 1189 | } 1190 | } 1191 | 1192 | /* update cpaSessHandle if need */ 1193 | if (0 != memcmp(&zstdSess->sessionSetupData, 1194 | &gProcess.qzstdInst[i].sessionSetupData, 1195 | sizeof(CpaDcSessionSetupData))) { 1196 | if (QZSTD_OK != QZSTD_cpaUpdateSess(zstdSess, i)) { 1197 | QZSTD_LOG(1, "Failed to update sess\n"); 1198 | rc = ZSTD_SEQUENCE_PRODUCER_ERROR; 1199 | goto exit; 1200 | } 1201 | } 1202 | 1203 | if (CPA_STATUS_SUCCESS != cpaDcLZ4SCompressBound(gProcess.dcInstHandle[i], 1204 | ZSTD_BLOCKSIZE_MAX, &intermediateBufLen)) { 1205 | QZSTD_LOG(1, "Failed to caculate compress bound\n"); 1206 | rc = ZSTD_SEQUENCE_PRODUCER_ERROR; 1207 | goto exit; 1208 | } 1209 | 1210 | /* Allocate intermediate buffer for storing lz4s format compressed by QAT */ 1211 | if (NULL == zstdSess->qatIntermediateBuf) { 1212 | zstdSess->qatIntermediateBuf = 1213 | (unsigned char *)QZSTD_calloc(1, intermediateBufLen, 1214 | zstdSess->reqPhyContMem); 1215 | if (NULL == zstdSess->qatIntermediateBuf) { 1216 | QZSTD_LOG(1, "Failed to allocate memory"); 1217 | rc = ZSTD_SEQUENCE_PRODUCER_ERROR; 1218 | goto exit; 1219 | } 1220 | } 1221 | 1222 | if (zstdSess->reqPhyContMem) { 1223 | memcpy(gProcess.qzstdInst[i].srcBuffer->pBuffers->pData, src, srcSize); 1224 | } else { 1225 | QZSTD_castConstPointer(&(gProcess.qzstdInst[i].srcBuffer->pBuffers->pData), 1226 | &src); 1227 | } 1228 | gProcess.qzstdInst[i].destBuffer->pBuffers->pData = 1229 | (Cpa8U *)zstdSess->qatIntermediateBuf; 1230 | 1231 | gProcess.qzstdInst[i].srcBuffer->pBuffers->dataLenInBytes = srcSize; 1232 | gProcess.qzstdInst[i].destBuffer->pBuffers->dataLenInBytes = 1233 | intermediateBufLen; 1234 | 1235 | memset(&opData, 0, sizeof(CpaDcOpData)); 1236 | opData.inputSkipData.skipMode = CPA_DC_SKIP_DISABLED; 1237 | opData.outputSkipData.skipMode = CPA_DC_SKIP_DISABLED; 1238 | opData.compressAndVerify = CPA_TRUE; 1239 | opData.flushFlag = CPA_DC_FLUSH_FINAL; 1240 | 1241 | gProcess.qzstdInst[i].res.checksum = 0; 1242 | 1243 | do { 1244 | /* Submit request to QAT */ 1245 | qrc = cpaDcCompressData2(gProcess.dcInstHandle[i], 1246 | gProcess.qzstdInst[i].cpaSessHandle, 1247 | gProcess.qzstdInst[i].srcBuffer, 1248 | gProcess.qzstdInst[i].destBuffer, &opData, 1249 | &gProcess.qzstdInst[i].res, (void *)&gProcess.qzstdInst[i]); 1250 | retry_cnt--; 1251 | } while (CPA_STATUS_RETRY == qrc && retry_cnt > 0); 1252 | 1253 | if (CPA_STATUS_SUCCESS != qrc) { 1254 | QZSTD_LOG(1, "Failed to submit request, status: %d\n", qrc); 1255 | rc = ZSTD_SEQUENCE_PRODUCER_ERROR; 1256 | goto error; 1257 | } 1258 | 1259 | gProcess.qzstdInst[i].seqNumIn++; 1260 | 1261 | (void)gettimeofday(&timeStart, NULL); 1262 | 1263 | do { 1264 | /* Poll responses */ 1265 | qrc = icp_sal_DcPollInstance(gProcess.dcInstHandle[i], 0); 1266 | (void)gettimeofday(&timeNow, NULL); 1267 | if (QZSTD_isTimeOut(timeStart, timeNow)) { 1268 | QZSTD_LOG(1, "Polling time out\n"); 1269 | break; 1270 | } 1271 | } while (CPA_STATUS_RETRY == qrc || (CPA_STATUS_SUCCESS == qrc && 1272 | gProcess.qzstdInst[i].seqNumIn != gProcess.qzstdInst[i].seqNumOut)); 1273 | 1274 | if (CPA_STATUS_FAIL == qrc) { 1275 | gProcess.qzstdInst[i].seqNumOut++; 1276 | QZSTD_LOG(1, "Polling failed, polling status: %d\n", qrc); 1277 | rc = ZSTD_SEQUENCE_PRODUCER_ERROR; 1278 | goto error; 1279 | } 1280 | 1281 | if (CPA_STATUS_RETRY == qrc) { 1282 | QZSTD_LOG(1, "Polling failed, polling status: %d\n", qrc); 1283 | rc = ZSTD_SEQUENCE_PRODUCER_ERROR; 1284 | goto error; 1285 | } 1286 | if (gProcess.qzstdInst[i].cbStatus == QZSTD_FAIL) { 1287 | QZSTD_LOG(1, "Error in dc callback, cbStatus: %d\n", 1288 | gProcess.qzstdInst[i].cbStatus); 1289 | rc = ZSTD_SEQUENCE_PRODUCER_ERROR; 1290 | goto error; 1291 | } 1292 | 1293 | if (gProcess.qzstdInst[i].res.consumed < srcSize || 1294 | gProcess.qzstdInst[i].res.produced == 0 || 1295 | gProcess.qzstdInst[i].res.produced > intermediateBufLen || 1296 | CPA_STATUS_SUCCESS != gProcess.qzstdInst[i].res.status) { 1297 | QZSTD_LOG(1, 1298 | "QAT result error, srcSize: %lu, consumed: %d, produced: %d, res.status:%d\n", 1299 | srcSize, gProcess.qzstdInst[i].res.consumed, gProcess.qzstdInst[i].res.produced, 1300 | gProcess.qzstdInst[i].res.status); 1301 | rc = ZSTD_SEQUENCE_PRODUCER_ERROR; 1302 | goto error; 1303 | } 1304 | QZSTD_LOG(2, "srcSize: %lu, consumed: %d, produced: %d\n", 1305 | srcSize, gProcess.qzstdInst[i].res.consumed, 1306 | gProcess.qzstdInst[i].res.produced); 1307 | 1308 | /* If source data is uncompressed, create one sequence */ 1309 | if (CPA_TRUE == gProcess.qzstdInst[i].res.dataUncompressed) { 1310 | outSeqs[0].litLength = srcSize; 1311 | outSeqs[0].offset = 0; 1312 | outSeqs[0].matchLength = 0; 1313 | rc = 1; 1314 | } else { 1315 | rc = QZSTD_decLz4s(outSeqs, outSeqsCapacity, zstdSess->qatIntermediateBuf, 1316 | gProcess.qzstdInst[i].res.produced); 1317 | } 1318 | if (rc >= (outSeqsCapacity - 1) || ZSTD_SEQUENCE_PRODUCER_ERROR == rc) { 1319 | QZSTD_LOG(1, "Decode error\n"); 1320 | rc = ZSTD_SEQUENCE_PRODUCER_ERROR; 1321 | goto error; 1322 | } 1323 | QZSTD_LOG(2, "Produced %lu sequences\n", rc); 1324 | 1325 | error: 1326 | /* reset pData */ 1327 | if (!zstdSess->reqPhyContMem) { 1328 | gProcess.qzstdInst[i].srcBuffer->pBuffers->pData = NULL; 1329 | } 1330 | gProcess.qzstdInst[i].destBuffer->pBuffers->pData = NULL; 1331 | 1332 | exit: 1333 | /* release QAT instance */ 1334 | QZSTD_releaseInstance(i); 1335 | return rc; 1336 | } 1337 | -------------------------------------------------------------------------------- /src/qatseqprod.h: -------------------------------------------------------------------------------- 1 | /*************************************************************************** 2 | * 3 | * BSD LICENSE 4 | * 5 | * Copyright(c) 2007-2023 Intel Corporation. All rights reserved. 6 | * All rights reserved. 7 | * 8 | * Redistribution and use in source and binary forms, with or without 9 | * modification, are permitted provided that the following conditions 10 | * are met: 11 | * 12 | * * Redistributions of source code must retain the above copyright 13 | * notice, this list of conditions and the following disclaimer. 14 | * * Redistributions in binary form must reproduce the above copyright 15 | * notice, this list of conditions and the following disclaimer in 16 | * the documentation and/or other materials provided with the 17 | * distribution. 18 | * * Neither the name of Intel Corporation nor the names of its 19 | * contributors may be used to endorse or promote products derived 20 | * from this software without specific prior written permission. 21 | * 22 | * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 23 | * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 24 | * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 25 | * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 26 | * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 27 | * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 28 | * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 29 | * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 30 | * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 31 | * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 32 | * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 33 | * 34 | ***************************************************************************/ 35 | #if defined (__cplusplus) 36 | extern "C" { 37 | #endif 38 | 39 | #ifndef QATSEQPROD_H 40 | #define QATSEQPROD_H 41 | 42 | #ifndef ZSTD_STATIC_LINKING_ONLY 43 | #define ZSTD_STATIC_LINKING_ONLY 44 | #endif 45 | #include "zstd.h" 46 | 47 | /** 48 | * Version 49 | */ 50 | #define QZSTD_VERSION "0.2.0" 51 | #define QZSTD_VERSION_MAJOR 0 52 | #define QZSTD_VERSION_MINOR 2 53 | #define QZSTD_VERSION_RELEASE 0 54 | #define QZSTD_VERSION_NUMBER (QZSTD_VERSION_MAJOR *100*100 + QZSTD_VERSION_MINOR *100 \ 55 | + QZSTD_VERSION_RELEASE) 56 | 57 | /** QZSTD_Status_e: 58 | * Error code indicates status 59 | */ 60 | typedef enum { 61 | QZSTD_OK = 0, /* Success */ 62 | QZSTD_STARTED = 1, /* QAT device started */ 63 | QZSTD_FAIL = -1, /* Unspecified error */ 64 | QZSTD_UNSUPPORTED = -2 /* Unsupport */ 65 | } QZSTD_Status_e; 66 | 67 | /** QZSTD_version: 68 | * Return the version of QAT Zstd Plugin. 69 | * 70 | * @retval const char* Version string. 71 | */ 72 | const char *QZSTD_version(void); 73 | 74 | /** qatSequenceProducer: 75 | * Block-level sequence producer with QAT 76 | * This implementation can be registered to zstd by ZSTD_registerSequenceProducer 77 | * for replacing internal block-level sequence producer. With this sequence producer, 78 | * zstd can offload the process of producing block-level sequences to QAT device. 79 | * 80 | * @param sequenceProducerState A pointer to a user-managed state for QAT sequence producer, 81 | * users need to call QZSTD_createSeqProdState to create it, and 82 | * call QZSTD_freeSeqProdState to free it. 83 | * @param outSeqs The output buffer for QAT sequence producer. The memory backing 84 | * outSeqs is managed by the CCtx. 85 | * @param outSeqsCapacity outSeqsCapacity is guaranteed >= ZSTD_sequenceBound(srcSize). 86 | * @param src An input buffer for the sequence producer to parse. 87 | * @param srcSize The size of input buffer which is guaranteed to be <= ZSTD_BLOCKSIZE_MAX. 88 | * @param dict Dict buffer for sequence producer to reference. Currently, it's a NULL pointer, 89 | * and will be supported in the future. 90 | * @param dictSize The size of dict. Currently, zstd will always pass zero into sequence producer. 91 | * @param compressionLevel Zstd compression level, only support L1-L12. 92 | * @param windowSize Representing the maximum allowed offset for sequences 93 | * 94 | * @retval size_t Return number of sequences QAT sequence producer produced 95 | * or error code: ZSTD_SEQUENCE_PRODUCER_ERROR. 96 | * *** LIMITATIONS *** 97 | * - Only support compression level from L1 to L12. 98 | * - ZSTD sequence producer only support zstd compression API which respect advanced parameters. 99 | * - The ZSTD_c_enableLongDistanceMatching cParam is not currently supported. Compression will fail 100 | * if it is enabled and tries to compress with qatsequenceproducer. 101 | * - Dictionaries are not currently supported. Compression will not fail if the user references 102 | * a dictionary, but the dictionary won't have any effect. 103 | * - Stream history is not currently supported. All advanced ZSTD compression APIs, including 104 | * streaming APIs, work with qatsequenceproducer, but each block is treated as an independent 105 | * chunk without history from previous blocks. 106 | * - Multi-threading within a single compression is not currently supported. In other words, 107 | * compression will fail if ZSTD_c_nbWorkers > 0 and an external sequence producer is registered. 108 | * Multi-threading across compressions is fine: simply create one CCtx per thread. 109 | */ 110 | size_t qatSequenceProducer( 111 | void *sequenceProducerState, ZSTD_Sequence *outSeqs, size_t outSeqsCapacity, 112 | const void *src, size_t srcSize, 113 | const void *dict, size_t dictSize, 114 | int compressionLevel, 115 | size_t windowSize 116 | ); 117 | 118 | /** QZSTD_startQatDevice: 119 | * Start QAT device 120 | * This function is used to initialize the QAT hardware. If qatSequenceProducer 121 | * is registered, the QAT device must also be started before the compression 122 | * work starts. 123 | * 124 | * @retval QZSTD_OK QAT device is fully successfully been started. 125 | * @retval QZSTD_STARTED QAT device is started, but the capability does not 126 | * meet the requirements. 127 | * @retval QZSTD_FAIL Failed to start QAT device. 128 | * @retval QZSTD_UNSUPPORTED QAT device or current configuration didn't support LZ4s and postprocessing. 129 | */ 130 | int QZSTD_startQatDevice(void); 131 | 132 | /** QZSTD_stopQatDevice: 133 | * Stop QAT device 134 | * This function is used to free hardware resources. Users need to call this 135 | * function after all compression jobs are finished. 136 | */ 137 | void QZSTD_stopQatDevice(void); 138 | 139 | /** QZSTD_createSeqProdState: 140 | * Create sequence producer state for qatSequenceProducer 141 | * The pointer returned by this function is required for registering qatSequenceProducer. 142 | * One ZSTD CCtx can share one sequence producer state, no need to reallocate for every 143 | * compression job. 144 | */ 145 | void *QZSTD_createSeqProdState(void); 146 | 147 | /** QZSTD_freeSeqProdState: 148 | * Free sequence producer state qatSequenceProducer used 149 | * After all compression jobs are finished, users must free the sequence producer state. 150 | */ 151 | void QZSTD_freeSeqProdState(void *sequenceProducerState); 152 | 153 | #endif /* QATSEQPROD_H */ 154 | 155 | #if defined (__cplusplus) 156 | } 157 | #endif 158 | -------------------------------------------------------------------------------- /test/Makefile: -------------------------------------------------------------------------------- 1 | # ####################################################################### 2 | # 3 | # BSD LICENSE 4 | # 5 | # Copyright(c) 2007-2023 Intel Corporation. All rights reserved. 6 | # All rights reserved. 7 | # 8 | # Redistribution and use in source and binary forms, with or without 9 | # modification, are permitted provided that the following conditions 10 | # are met: 11 | # 12 | # * Redistributions of source code must retain the above copyright 13 | # notice, this list of conditions and the following disclaimer. 14 | # * Redistributions in binary form must reproduce the above copyright 15 | # notice, this list of conditions and the following disclaimer in 16 | # the documentation and/or other materials provided with the 17 | # distribution. 18 | # * Neither the name of Intel Corporation nor the names of its 19 | # contributors may be used to endorse or promote products derived 20 | # from this software without specific prior written permission. 21 | # 22 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 23 | # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 24 | # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 25 | # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 26 | # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 27 | # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 28 | # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 29 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 30 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 31 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 32 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 33 | # 34 | # ####################################################################### 35 | LIB = ../src 36 | 37 | LDFLAGS = $(LIB)/libqatseqprod.a -I$(LIB) 38 | 39 | ifneq ($(ICP_ROOT), ) 40 | LDFLAGS += -lqat_s -lusdm_drv_s -Wl,-rpath,$(ICP_ROOT)/build -L$(ICP_ROOT)/build 41 | else 42 | LDFLAGS += -lqat -lusdm 43 | endif 44 | 45 | ifdef ZSTDLIB 46 | LDFLAGS += $(ZSTDLIB)/libzstd.a 47 | else 48 | ZSTDLIB := $(shell find /usr /lib /local -name 'libzstd.a' 2>/dev/null | head -n 1) 49 | ZSTDLIB ?= /usr/local/lib/libzstd.a 50 | ifneq ("$(wildcard $(ZSTDLIB))","") 51 | LDFLAGS += $(ZSTDLIB) 52 | else 53 | $(error libzstd.a not found, please install libzstd or specify the path manually) 54 | endif 55 | endif 56 | 57 | default: test benchmark 58 | 59 | all: test benchmark 60 | 61 | test: test.c 62 | $(Q)$(MAKE) -C $(LIB) 63 | $(CC) $^ $(CFLAGS) $(LDFLAGS) -o $@ 64 | 65 | benchmark: benchmark.c 66 | $(Q)$(MAKE) -C $(LIB) 67 | $(CC) $(CFLAGS) $^ $(LDFLAGS) -o $@ -lpthread 68 | 69 | clean: 70 | $(Q)$(MAKE) -C $(LIB) $@ 71 | $(RM) test benchmark 72 | -------------------------------------------------------------------------------- /test/benchmark.c: -------------------------------------------------------------------------------- 1 | /*************************************************************************** 2 | * 3 | * BSD LICENSE 4 | * 5 | * Copyright(c) 2023 Intel Corporation. All rights reserved. 6 | * All rights reserved. 7 | * 8 | * Redistribution and use in source and binary forms, with or without 9 | * modification, are permitted provided that the following conditions 10 | * are met: 11 | * 12 | * * Redistributions of source code must retain the above copyright 13 | * notice, this list of conditions and the following disclaimer. 14 | * * Redistributions in binary form must reproduce the above copyright 15 | * notice, this list of conditions and the following disclaimer in 16 | * the documentation and/or other materials provided with the 17 | * distribution. 18 | * * Neither the name of Intel Corporation nor the names of its 19 | * contributors may be used to endorse or promote products derived 20 | * from this software without specific prior written permission. 21 | * 22 | * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 23 | * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 24 | * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 25 | * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 26 | * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 27 | * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 28 | * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 29 | * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 30 | * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 31 | * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 32 | * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 33 | * 34 | ***************************************************************************/ 35 | 36 | #include 37 | #include 38 | #include 39 | #include 40 | #include 41 | #include 42 | #include 43 | #include 44 | #include 45 | #include 46 | 47 | #ifndef ZSTD_STATIC_LINKING_ONLY 48 | #define ZSTD_STATIC_LINKING_ONLY 49 | #endif 50 | #include "zstd.h" 51 | #include "zstd_errors.h" 52 | #include "qatseqprod.h" 53 | 54 | #define NANOSEC (1000000000ULL) /* 1 second */ 55 | #define NANOUSEC (1000) /* 1 usec */ 56 | #define MB (1000000) /* 1MB */ 57 | #define BUCKET_NUM 200 58 | #define DEFAULT_CHUNK_SIZE (32 * 1024) 59 | #define ZSTD_AUTO 0 60 | #define ZSTD_ENABLED 1 61 | #define ZSTD_DISABLED 2 62 | 63 | 64 | #ifndef MIN 65 | #define MIN(a,b) ((a)<(b)?(a):(b)) 66 | #endif 67 | 68 | #define DISPLAY(...) fprintf(stderr, __VA_ARGS__) 69 | 70 | #define GETTIME(now) {clock_gettime(CLOCK_MONOTONIC, &now);}; 71 | #define GETDIFFTIME(start_ticks, end_ticks) (1000000000ULL*( end_ticks.tv_sec - start_ticks.tv_sec ) + ( end_ticks.tv_nsec - start_ticks.tv_nsec )) 72 | 73 | 74 | typedef struct { 75 | size_t chunkSize; /* the max chunk size of ZSTD_compress2 */ 76 | size_t srcSize; /* Input file size */ 77 | unsigned cLevel; /* Compression Level 1 - 12 */ 78 | unsigned nbIterations; /* Number test loops, default is 1 */ 79 | char benchMode; /* 0: software compression, 1: QAT compression*/ 80 | char searchForExternalRepcodes; /* 0: auto 1: enable, 2: disable */ 81 | const unsigned char *srcBuffer; /* Input data point */ 82 | } threadArgs_t; 83 | 84 | typedef struct { 85 | size_t bucketValue[BUCKET_NUM]; 86 | size_t bucket[BUCKET_NUM]; 87 | size_t maxBucketValue; 88 | size_t minBucketValue; 89 | int bucketCount; 90 | size_t num; 91 | size_t sum; 92 | size_t min; 93 | size_t max; 94 | } HistogramStat_t; 95 | 96 | static HistogramStat_t compHistogram; 97 | static pthread_barrier_t g_threadBarrier1, g_threadBarrier2; 98 | static size_t g_threadNum = 0; 99 | 100 | static void initHistorgram(HistogramStat_t *historgram) 101 | { 102 | historgram->bucketValue[0] = 1000; 103 | historgram->minBucketValue = historgram->bucketValue[0]; 104 | historgram->bucketCount = 1; 105 | for (int i = 1; i < BUCKET_NUM; i++) { 106 | if (historgram->bucketValue[i] > 0XFFFFFFFF) { 107 | break; 108 | } 109 | historgram->bucketValue[i] = historgram->bucketValue[i - 1] * 1.05; 110 | historgram->maxBucketValue = historgram->bucketValue[i]; 111 | historgram->bucketCount++; 112 | } 113 | memset(historgram->bucket, 0, sizeof(size_t) * BUCKET_NUM); 114 | historgram->num = 0; 115 | historgram->sum = 0; 116 | historgram->min = 0xFFFFFFFF; 117 | historgram->max = 0; 118 | } 119 | 120 | static int getBucketIndex(HistogramStat_t *historgram, size_t value) 121 | { 122 | for (int bucketIndex = 0; bucketIndex < BUCKET_NUM; bucketIndex++) { 123 | if (value < historgram->bucketValue[bucketIndex]) { 124 | return bucketIndex; 125 | } 126 | } 127 | return BUCKET_NUM - 1; 128 | } 129 | 130 | static void bucketAdd(HistogramStat_t *historgram, size_t value) 131 | { 132 | int bucketIndex = getBucketIndex(historgram, value); 133 | __sync_fetch_and_add(&historgram->bucket[bucketIndex], 1); 134 | 135 | __sync_fetch_and_add(&historgram->sum, value); 136 | __sync_fetch_and_add(&historgram->num, 1); 137 | if (value < historgram->min) 138 | __sync_lock_test_and_set(&historgram->min, value); 139 | if (value > historgram->max) 140 | __sync_lock_test_and_set(&historgram->max, value); 141 | } 142 | 143 | static double percentile(HistogramStat_t *historgram, double p) 144 | { 145 | double threshold = historgram->num * (p / 100.0); 146 | size_t cumulative_sum = 0; 147 | for (int index = 0; index < historgram->bucketCount; index++) { 148 | size_t bucket_count = historgram->bucket[index]; 149 | cumulative_sum += bucket_count; 150 | if (cumulative_sum >= threshold) { 151 | size_t left_point = (index == 0) ? 0 : historgram->bucketValue[index - 1]; 152 | size_t right_point = historgram->bucketValue[index]; 153 | size_t left_sum = cumulative_sum - bucket_count; 154 | size_t right_sum = cumulative_sum; 155 | double pos = 0; 156 | size_t right_left_diff = right_sum - left_sum; 157 | if (right_left_diff != 0) { 158 | pos = (threshold - left_sum) / right_left_diff; 159 | } 160 | double r = left_point + (right_point - left_point) * pos; 161 | if (r < historgram->min) 162 | r = historgram->min; 163 | if (r > historgram->max) 164 | r = historgram->max; 165 | return r; 166 | } 167 | } 168 | return historgram->max; 169 | } 170 | 171 | static int usage(const char *exe) 172 | { 173 | DISPLAY("Usage:\n"); 174 | DISPLAY(" %s [arg] filename\n", exe); 175 | DISPLAY("Options:\n"); 176 | DISPLAY(" -t# Set maximum threads [1 - 128] (default: 1)\n"); 177 | DISPLAY(" -l# Set iteration loops [1 - 1000000](default: 1)\n"); 178 | DISPLAY(" -c# Set chunk size (default: 32K)\n"); 179 | DISPLAY(" -E# Auto/enable/disable searchForExternalRepcodes(0: auto; 1: enable; 2: disable; default: auto)\n"); 180 | DISPLAY(" -L# Set compression level [1 - 12] (default: 1)\n"); 181 | DISPLAY(" -m# Benchmark mode, 0: software compression; 1:QAT compression(default: 1) \n"); 182 | DISPLAY(" -h/H Print this help message\n"); 183 | return 0; 184 | } 185 | 186 | /* this function to convert string to unsigned int, 187 | * the string MUST BE starting with numeric and can be 188 | * end with "K" or "M". Such as: 189 | * if input string is "128K" and output will be 131072. 190 | * if input string is "65536" and output will be 65536. 191 | */ 192 | static unsigned stringToU32(const char **s) 193 | { 194 | unsigned value = 0; 195 | while ((**s >= '0') && (**s <= '9')) { 196 | if (value > ((((unsigned)(-1)) / 10) - 1)) { 197 | DISPLAY("ERROR: numeric value is too large\n"); 198 | exit(1); 199 | } 200 | value *= 10; 201 | value += (unsigned)(**s - '0'); 202 | (*s)++ ; 203 | } 204 | if ((**s == 'K') || (**s == 'M')) { 205 | if (value > ((unsigned)(-1)) >> 10) { 206 | DISPLAY("ERROR: numeric value is too large\n"); 207 | exit(1); 208 | } 209 | value <<= 10; 210 | if (**s == 'M') { 211 | if (value > ((unsigned)(-1)) >> 10) { 212 | DISPLAY("ERROR: numeric value is too large\n"); 213 | exit(1); 214 | } 215 | value <<= 10; 216 | } 217 | (*s)++; 218 | } 219 | return value; 220 | } 221 | 222 | void *benchmark(void *args) 223 | { 224 | threadArgs_t *threadArgs = (threadArgs_t *)args; 225 | size_t rc = 0, threadNum; 226 | unsigned loops; 227 | int verifyResult = 0; /* 1: pass, 0: fail */ 228 | size_t *chunkSizes = NULL; /* The array of chunk size */ 229 | size_t *compSizes = NULL; /* The array of compressed size */ 230 | size_t nanosec = 0; 231 | size_t compNanosecSum = 0, decompNanosecSum = 0; 232 | double compSpeed = 0, decompSpeed = 0, ratio = 0; 233 | size_t csCount, nbChunk, destSize, cSize, dcSize; 234 | struct timespec startTicks, endTicks; 235 | unsigned char *destBuffer = NULL, *decompBuffer = NULL; 236 | const unsigned char *srcBuffer = threadArgs->srcBuffer; 237 | size_t srcSize = threadArgs->srcSize; 238 | size_t chunkSize = threadArgs->chunkSize; 239 | unsigned nbIterations = threadArgs->nbIterations; 240 | unsigned cLevel = threadArgs->cLevel; 241 | ZSTD_CCtx *const zc = ZSTD_createCCtx(); 242 | ZSTD_DCtx *const zdc = ZSTD_createDCtx(); 243 | void *matchState = NULL; 244 | int setUpStatus = 0, compressStatus = 0; 245 | 246 | csCount = srcSize / chunkSize + (srcSize % chunkSize ? 1 : 0); 247 | chunkSizes = (size_t *)malloc(csCount * sizeof(size_t)); 248 | compSizes = (size_t *)malloc(csCount * sizeof(size_t)); 249 | assert(chunkSizes && compSizes); 250 | size_t tmpSize = srcSize; 251 | for (nbChunk = 0; nbChunk < csCount; nbChunk++) { 252 | chunkSizes[nbChunk] = MIN(tmpSize, chunkSize); 253 | tmpSize -= chunkSizes[nbChunk]; 254 | } 255 | 256 | destSize = ZSTD_compressBound(srcSize); 257 | destBuffer = (unsigned char *)malloc(destSize); 258 | decompBuffer = (unsigned char *)malloc(srcSize); 259 | assert(destBuffer != NULL); 260 | 261 | if (threadArgs->benchMode == 1) { 262 | QZSTD_startQatDevice(); 263 | matchState = QZSTD_createSeqProdState(); 264 | ZSTD_registerSequenceProducer(zc, matchState, qatSequenceProducer); 265 | } else { 266 | ZSTD_registerSequenceProducer(zc, NULL, NULL); 267 | } 268 | 269 | if (threadArgs->searchForExternalRepcodes == ZSTD_ENABLED) { 270 | rc = ZSTD_CCtx_setParameter(zc, ZSTD_c_searchForExternalRepcodes, 271 | ZSTD_ps_enable); 272 | } else if (threadArgs->searchForExternalRepcodes == ZSTD_DISABLED) { 273 | rc = ZSTD_CCtx_setParameter(zc, ZSTD_c_searchForExternalRepcodes, 274 | ZSTD_ps_disable); 275 | } else { 276 | rc = ZSTD_CCtx_setParameter(zc, ZSTD_c_searchForExternalRepcodes, ZSTD_ps_auto); 277 | } 278 | if (ZSTD_isError(rc)) { 279 | DISPLAY("Fail to set parameter ZSTD_c_searchForExternalRepcodes\n"); 280 | goto setupend; 281 | } 282 | 283 | rc = ZSTD_CCtx_setParameter(zc, ZSTD_c_compressionLevel, cLevel); 284 | if (ZSTD_isError(rc)) { 285 | DISPLAY("Fail to set parameter ZSTD_c_compressionLevel\n"); 286 | goto setupend; 287 | } 288 | 289 | setUpStatus = 1; 290 | 291 | setupend: 292 | 293 | /* Waiting all threads */ 294 | pthread_barrier_wait(&g_threadBarrier1); 295 | if (!setUpStatus) { 296 | goto compressend; 297 | } 298 | 299 | /* Start compression benchmark */ 300 | for (loops = 0; loops < nbIterations; loops++) { 301 | unsigned char *tmpDestBuffer = destBuffer; 302 | const unsigned char *tmpSrcBuffer = srcBuffer; 303 | size_t tmpDestSize = destSize; 304 | for (nbChunk = 0; nbChunk < csCount; nbChunk++) { 305 | GETTIME(startTicks); 306 | cSize = ZSTD_compress2(zc, tmpDestBuffer, tmpDestSize, tmpSrcBuffer, 307 | chunkSizes[nbChunk]); 308 | GETTIME(endTicks); 309 | if (ZSTD_isError(cSize)) { 310 | DISPLAY("Compress failed\n"); 311 | goto compressend; 312 | } 313 | tmpDestBuffer += cSize; 314 | tmpDestSize -= cSize; 315 | tmpSrcBuffer += chunkSizes[nbChunk]; 316 | compSizes[nbChunk] = cSize; 317 | nanosec = GETDIFFTIME(startTicks, endTicks); 318 | bucketAdd(&compHistogram, nanosec); 319 | compNanosecSum += nanosec; 320 | } 321 | } 322 | 323 | cSize = 0; 324 | for (nbChunk = 0; nbChunk < csCount; nbChunk++) { 325 | cSize += compSizes[nbChunk]; 326 | } 327 | 328 | /* Verify the compression result */ 329 | rc = ZSTD_decompress(decompBuffer, srcSize, destBuffer, cSize); 330 | if (rc != srcSize) { 331 | DISPLAY("Decompressed size is not equal to source size\n"); 332 | goto compressend; 333 | } 334 | /* Compare original buffer with decompress output */ 335 | if (!memcmp(decompBuffer, srcBuffer, srcSize)) { 336 | verifyResult = 1; 337 | } else { 338 | verifyResult = 0; 339 | } 340 | 341 | compressStatus = 1; 342 | 343 | compressend: 344 | 345 | pthread_barrier_wait(&g_threadBarrier2); 346 | if (!setUpStatus || !compressStatus) { 347 | goto exit; 348 | } 349 | /* Start decompression benchmark */ 350 | for (loops = 0; loops < nbIterations; loops++) { 351 | unsigned char *tmpDestBuffer = decompBuffer; 352 | const unsigned char *tmpSrcBuffer = destBuffer; 353 | size_t tmpDestSize = srcSize; 354 | for (nbChunk = 0; nbChunk < csCount; nbChunk++) { 355 | GETTIME(startTicks); 356 | dcSize = ZSTD_decompressDCtx(zdc, tmpDestBuffer, tmpDestSize, tmpSrcBuffer, 357 | compSizes[nbChunk]); 358 | GETTIME(endTicks); 359 | if (ZSTD_isError(dcSize)) { 360 | DISPLAY("Decompress failed\n"); 361 | goto exit; 362 | } 363 | tmpDestBuffer += dcSize; 364 | tmpDestSize -= dcSize; 365 | tmpSrcBuffer += compSizes[nbChunk]; 366 | nanosec = GETDIFFTIME(startTicks, endTicks); 367 | decompNanosecSum += nanosec; 368 | } 369 | } 370 | 371 | /*Get current thread num */ 372 | threadNum = __sync_add_and_fetch(&g_threadNum, 1); 373 | 374 | ratio = (double) cSize / (double)srcSize; 375 | compSpeed = (double)(srcSize * nbIterations) / ((double)compNanosecSum / 376 | NANOSEC); 377 | decompSpeed = (double)(srcSize * nbIterations) / ((double)decompNanosecSum / 378 | NANOSEC); 379 | DISPLAY("Thread %lu: Compression: %lu -> %lu, Throughput: Comp: %5.f MB/s, Decomp: %5.f MB/s, Compression Ratio: %2.2f%%, %s\n", 380 | threadNum, srcSize, cSize, (double) compSpeed / MB, (double) decompSpeed / MB, 381 | ratio * 100, 382 | verifyResult ? "PASS" : "FAIL"); 383 | exit: 384 | ZSTD_freeCCtx(zc); 385 | ZSTD_freeDCtx(zdc); 386 | if (threadArgs->benchMode == 1 && matchState) { 387 | QZSTD_freeSeqProdState(matchState); 388 | } 389 | if (chunkSizes) { 390 | free(chunkSizes); 391 | } 392 | if (compSizes) { 393 | free(compSizes); 394 | } 395 | if (destBuffer) { 396 | free(destBuffer); 397 | } 398 | if (decompBuffer) { 399 | free(decompBuffer); 400 | } 401 | return NULL; 402 | } 403 | 404 | int main(int argc, const char **argv) 405 | { 406 | int argNb, threadNb; 407 | int nbThreads = 1; 408 | pthread_t threads[2048]; 409 | size_t srcSize, bytesRead; 410 | unsigned char *srcBuffer = NULL; 411 | const char *fileName = NULL; 412 | int inputFile = -1; 413 | threadArgs_t threadArgs; 414 | 415 | if (argc < 2) 416 | return usage(argv[0]); 417 | 418 | /* Set default value */ 419 | threadArgs.chunkSize = DEFAULT_CHUNK_SIZE; 420 | threadArgs.nbIterations = 1; 421 | threadArgs.cLevel = 1; 422 | threadArgs.benchMode = 1; 423 | threadArgs.searchForExternalRepcodes = ZSTD_AUTO; 424 | 425 | for (argNb = 1; argNb < argc; argNb++) { 426 | const char *arg = argv[argNb]; 427 | if (arg[0] == '-') { 428 | arg++; 429 | while (arg[0] != 0) { 430 | switch (arg[0]) { 431 | /* Display help message */ 432 | case 'h': 433 | case 'H': 434 | return usage(argv[0]); 435 | /* Set maximum threads */ 436 | case 't': 437 | arg++; 438 | nbThreads = stringToU32(&arg); 439 | if (nbThreads > 2048) { 440 | DISPLAY("Invalid thread parameter, maximum is 2048\n"); 441 | return -1; 442 | } 443 | break; 444 | /* Set chunk size */ 445 | case 'c': 446 | arg++; 447 | threadArgs.chunkSize = stringToU32(&arg); 448 | break; 449 | /* Set iterations */ 450 | case 'l': 451 | arg++; 452 | threadArgs.nbIterations = stringToU32(&arg); 453 | break; 454 | /* Set benchmark mode */ 455 | case 'm': 456 | arg++; 457 | threadArgs.benchMode = stringToU32(&arg); 458 | break; 459 | /* Set searchForExternalRepcodes */ 460 | case 'E': 461 | arg++; 462 | threadArgs.searchForExternalRepcodes = stringToU32(&arg); 463 | if (threadArgs.searchForExternalRepcodes != ZSTD_AUTO && 464 | threadArgs.searchForExternalRepcodes != ZSTD_ENABLED && 465 | threadArgs.searchForExternalRepcodes != ZSTD_DISABLED) { 466 | DISPLAY("Invalid searchForExternalRepcodes parameter\n"); 467 | return usage(argv[0]); 468 | } 469 | break; 470 | /* Set compression level */ 471 | case 'L': 472 | arg++; 473 | threadArgs.cLevel = stringToU32(&arg); 474 | break; 475 | /* Unknown argument */ 476 | default : 477 | return usage(argv[0]); 478 | } 479 | } 480 | continue; 481 | } 482 | if (!fileName) { 483 | fileName = arg; 484 | continue; 485 | } 486 | } 487 | if (!fileName) { 488 | return usage(argv[0]); 489 | } 490 | 491 | /* Load input file */ 492 | inputFile = open(fileName, O_RDONLY); 493 | if (inputFile < 0) { 494 | DISPLAY("Cannot open input file: %s\n", fileName); 495 | return -1; 496 | } 497 | srcSize = lseek(inputFile, 0, SEEK_END); 498 | lseek(inputFile, 0, SEEK_SET); 499 | srcBuffer = (unsigned char *)malloc(srcSize); 500 | assert(srcBuffer != NULL); 501 | 502 | bytesRead = 0; 503 | while (bytesRead != srcSize) { 504 | bytesRead += read(inputFile, srcBuffer + bytesRead, srcSize - bytesRead); 505 | } 506 | assert(bytesRead == srcSize); 507 | 508 | threadArgs.srcBuffer = srcBuffer; 509 | threadArgs.srcSize = srcSize; 510 | initHistorgram(&compHistogram); 511 | 512 | pthread_barrier_init(&g_threadBarrier1, NULL, nbThreads); 513 | pthread_barrier_init(&g_threadBarrier2, NULL, nbThreads); 514 | for (threadNb = 0; threadNb < nbThreads; threadNb++) { 515 | pthread_create(&threads[threadNb], NULL, benchmark, &threadArgs); 516 | } 517 | 518 | for (threadNb = 0; threadNb < nbThreads; threadNb++) { 519 | pthread_join(threads[threadNb], NULL); 520 | } 521 | 522 | if (compHistogram.num != 0) { 523 | /* Display Latency statistics */ 524 | DISPLAY("-----------------------------------------------------------\n"); 525 | DISPLAY("Latency Percentiles: P25: %4.2f us, P50: %4.2f us, P75: %4.2f us, P99: %4.2f us, Avg: %4.2f us\n", 526 | percentile(&compHistogram, 25) / NANOUSEC, 527 | percentile(&compHistogram, 50) / NANOUSEC, 528 | percentile(&compHistogram, 75) / NANOUSEC, 529 | percentile(&compHistogram, 99) / NANOUSEC, 530 | (double)(compHistogram.sum / compHistogram.num / NANOUSEC)); 531 | 532 | #ifdef DISPLAY_HISTOGRAM 533 | DISPLAY("Latency histogram(nanosec): count: %lu\n", compHistogram.num); 534 | size_t cumulativeSum = 0; 535 | for (int i = 0; i < compHistogram.bucketCount; i++) { 536 | if (compHistogram.bucket[i] != 0) { 537 | cumulativeSum += compHistogram.bucket[i]; 538 | DISPLAY("[%10lu, %10lu] %10lu %7.2f%% %7.2f%%\n", 539 | i == 0 ? 0 : compHistogram.bucketValue[i - 1], compHistogram.bucketValue[i], 540 | compHistogram.bucket[i], 541 | (double)compHistogram.bucket[i] * 100 / compHistogram.num, 542 | (double)cumulativeSum * 100 / compHistogram.num); 543 | } 544 | } 545 | #endif 546 | } 547 | 548 | pthread_barrier_destroy(&g_threadBarrier1); 549 | pthread_barrier_destroy(&g_threadBarrier2); 550 | QZSTD_stopQatDevice(); 551 | close(inputFile); 552 | free(srcBuffer); 553 | return 0; 554 | } 555 | -------------------------------------------------------------------------------- /test/fuzzing/Makefile: -------------------------------------------------------------------------------- 1 | # ####################################################################### 2 | # 3 | # BSD LICENSE 4 | # 5 | # Copyright(c) 2007-2023 Intel Corporation. All rights reserved. 6 | # All rights reserved. 7 | # 8 | # Redistribution and use in source and binary forms, with or without 9 | # modification, are permitted provided that the following conditions 10 | # are met: 11 | # 12 | # * Redistributions of source code must retain the above copyright 13 | # notice, this list of conditions and the following disclaimer. 14 | # * Redistributions in binary form must reproduce the above copyright 15 | # notice, this list of conditions and the following disclaimer in 16 | # the documentation and/or other materials provided with the 17 | # distribution. 18 | # * Neither the name of Intel Corporation nor the names of its 19 | # contributors may be used to endorse or promote products derived 20 | # from this software without specific prior written permission. 21 | # 22 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 23 | # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 24 | # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 25 | # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 26 | # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 27 | # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 28 | # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 29 | # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 30 | # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 31 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 32 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 33 | # 34 | # ####################################################################### 35 | LIB = ../../src 36 | 37 | CC = clang 38 | 39 | default: qatseqprodfuzzer.o 40 | 41 | all: qatseqprodfuzzer.o 42 | 43 | ifneq ($(ICP_ROOT), ) 44 | QATFLAGS = -I$(ICP_ROOT)/quickassist/include \ 45 | -I$(ICP_ROOT)/quickassist/include/dc \ 46 | -I$(ICP_ROOT)/quickassist/lookaside/access_layer/include \ 47 | -I$(ICP_ROOT)/quickassist/utilities/libusdm_drv 48 | endif 49 | else 50 | QATFLAGS = -DINTREE 51 | endif 52 | 53 | ifdef ZSTDLIB 54 | CFLAGS += -I$(ZSTDLIB) 55 | endif 56 | 57 | 58 | CFLAGS += -g -fno-omit-frame-pointer -fsanitize=undefined,address,fuzzer 59 | DEBUGLEVEL ?=0 60 | DEBUGFLAGS += -DDEBUGLEVEL=$(DEBUGLEVEL) 61 | 62 | qatseqprodfuzzer.o: $(LIB)/qatseqprod.c 63 | $(CC) -c $(CFLAGS) $(QATFLAGS) $(DEBUGFLAGS) $^ -o qatseqprod.o 64 | $(CC) -c $(CFLAGS) qatseqprodfuzzer.c -o _qatseqprodfuzzer.o 65 | ld -r qatseqprod.o _qatseqprodfuzzer.o -o $@ 66 | 67 | clean: 68 | $(RM) *.o 69 | -------------------------------------------------------------------------------- /test/fuzzing/README.md: -------------------------------------------------------------------------------- 1 | Fuzzing test for QAT ZSTD Plugin 2 | ============================== 3 | 4 | Zstrandard*(ZSTD*) provides an interface for external sequence producer to do fuzzing test: [`fuzz_third_party_seq_prod.h`][1] after [1.5.5][2]. 5 | 6 | **Steps to run fuzzing test for QAT ZSTD Plugin** 7 | 8 | ```bash 9 | # Compile qatseqprodfuzzer.o with clang 10 | cd test/fuzzing 11 | make qatseqprodfuzzer.o 12 | 13 | # Build and run fuzzing targets 14 | git clone https://github.com/facebook/zstd.git 15 | cd tests/fuzz 16 | make corpora 17 | python3 ./fuzz.py build all --custom-seq-prod=*/qatseqprodfuzzer.o --enable-fuzzer --enable-asan --enable-ubsan --cc clang --cxx clang++ --ldflags=-lqat_s 18 | python3 ./fuzz.py libfuzzer simple_round_trip 19 | python3 ./fuzz.py libfuzzer stream_round_trip 20 | python3 ./fuzz.py libfuzzer dictionary_round_trip 21 | python3 ./fuzz.py libfuzzer block_round_trip 22 | python3 ./fuzz.py libfuzzer decompress_dstSize_tooSmall 23 | python3 ./fuzz.py libfuzzer dictionary_decompress 24 | python3 ./fuzz.py libfuzzer dictionary_loader 25 | python3 ./fuzz.py libfuzzer dictionary_stream_round_trip 26 | python3 ./fuzz.py libfuzzer raw_dictionary_round_trip 27 | python3 ./fuzz.py libfuzzer sequence_compression_api 28 | python3 ./fuzz.py libfuzzer simple_compress 29 | ``` 30 | 31 | [1]:https://github.com/facebook/zstd/blob/dev/tests/fuzz/fuzz_third_party_seq_prod.h 32 | [2]:https://github.com/facebook/zstd/releases/tag/v1.5.5 -------------------------------------------------------------------------------- /test/fuzzing/qatseqprodfuzzer.c: -------------------------------------------------------------------------------- 1 | /*************************************************************************** 2 | * 3 | * BSD LICENSE 4 | * 5 | * Copyright(c) 2007-2023 Intel Corporation. All rights reserved. 6 | * All rights reserved. 7 | * 8 | * Redistribution and use in source and binary forms, with or without 9 | * modification, are permitted provided that the following conditions 10 | * are met: 11 | * 12 | * * Redistributions of source code must retain the above copyright 13 | * notice, this list of conditions and the following disclaimer. 14 | * * Redistributions in binary form must reproduce the above copyright 15 | * notice, this list of conditions and the following disclaimer in 16 | * the documentation and/or other materials provided with the 17 | * distribution. 18 | * * Neither the name of Intel Corporation nor the names of its 19 | * contributors may be used to endorse or promote products derived 20 | * from this software without specific prior written permission. 21 | * 22 | * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 23 | * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 24 | * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 25 | * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 26 | * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 27 | * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 28 | * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 29 | * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 30 | * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 31 | * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 32 | * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 33 | * 34 | ***************************************************************************/ 35 | 36 | #include 37 | #include 38 | #include 39 | #include "qatseqprod.h" 40 | 41 | size_t FUZZ_seqProdSetup(void) 42 | { 43 | return QZSTD_startQatDevice(); 44 | } 45 | 46 | size_t FUZZ_seqProdTearDown(void) 47 | { 48 | return 0; 49 | } 50 | 51 | void *FUZZ_createSeqProdState(void) 52 | { 53 | return QZSTD_createSeqProdState(); 54 | } 55 | 56 | size_t FUZZ_freeSeqProdState(void *state) 57 | { 58 | QZSTD_freeSeqProdState(state); 59 | return 0; 60 | } 61 | 62 | size_t FUZZ_thirdPartySeqProd( 63 | void *sequenceProducerState, 64 | ZSTD_Sequence *outSeqs, size_t outSeqsCapacity, 65 | const void *src, size_t srcSize, 66 | const void *dict, size_t dictSize, 67 | int compressionLevel, 68 | size_t windowSize 69 | ) 70 | { 71 | return qatSequenceProducer(sequenceProducerState, outSeqs, 72 | outSeqsCapacity, src, srcSize, dict, dictSize, 73 | compressionLevel, windowSize); 74 | } -------------------------------------------------------------------------------- /test/test.c: -------------------------------------------------------------------------------- 1 | /*************************************************************************** 2 | * 3 | * BSD LICENSE 4 | * 5 | * Copyright(c) 2007-2023 Intel Corporation. All rights reserved. 6 | * All rights reserved. 7 | * 8 | * Redistribution and use in source and binary forms, with or without 9 | * modification, are permitted provided that the following conditions 10 | * are met: 11 | * 12 | * * Redistributions of source code must retain the above copyright 13 | * notice, this list of conditions and the following disclaimer. 14 | * * Redistributions in binary form must reproduce the above copyright 15 | * notice, this list of conditions and the following disclaimer in 16 | * the documentation and/or other materials provided with the 17 | * distribution. 18 | * * Neither the name of Intel Corporation nor the names of its 19 | * contributors may be used to endorse or promote products derived 20 | * from this software without specific prior written permission. 21 | * 22 | * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 23 | * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 24 | * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 25 | * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 26 | * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 27 | * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 28 | * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 29 | * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 30 | * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 31 | * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 32 | * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 33 | * 34 | ***************************************************************************/ 35 | 36 | #include 37 | #include 38 | #include 39 | #include 40 | #include 41 | #include 42 | #include 43 | #include 44 | 45 | #include "qatseqprod.h" 46 | 47 | #ifndef ZSTD_STATIC_LINKING_ONLY 48 | #define ZSTD_STATIC_LINKING_ONLY 49 | #endif 50 | #include "zstd.h" 51 | #include "zstd_errors.h" 52 | 53 | int main(int argc, char *argv[]) 54 | { 55 | char *inputFileName = NULL; 56 | int inputFile = -1; 57 | struct stat inputFileStat; 58 | long inputFileSize = 0; 59 | long dstBufferSize = 0; 60 | unsigned char *srcBuffer = NULL; 61 | unsigned char *dstBuffer = NULL; 62 | unsigned char *decompBuffer = NULL; 63 | unsigned long bytesRead = 0; 64 | size_t cSize = 0; 65 | size_t res = 0; 66 | ZSTD_CCtx *const zc = ZSTD_createCCtx(); 67 | QZSTD_startQatDevice(); 68 | void *sequenceProducerState = QZSTD_createSeqProdState(); 69 | 70 | if (argc != 2) { 71 | printf("Usage: test \n"); 72 | return 1; 73 | } 74 | 75 | inputFileName = argv[1]; 76 | inputFile = open(inputFileName, O_RDONLY); 77 | if (inputFile < 0) { 78 | printf("Cannot open input file: %s\n", inputFileName); 79 | return 1; 80 | } 81 | if (fstat(inputFile, &inputFileStat)) { 82 | printf("Cannot get file stat\n"); 83 | close(inputFile); 84 | return 1; 85 | } 86 | 87 | /* get input file size */ 88 | inputFileSize = lseek(inputFile, 0, SEEK_END); 89 | lseek(inputFile, 0, SEEK_SET); 90 | dstBufferSize = ZSTD_compressBound(inputFileSize); 91 | 92 | srcBuffer = (unsigned char *)malloc(inputFileSize); 93 | assert(srcBuffer != NULL); 94 | dstBuffer = (unsigned char *)malloc(dstBufferSize); 95 | assert(dstBuffer != NULL); 96 | 97 | bytesRead = read(inputFile, srcBuffer, inputFileSize); 98 | 99 | decompBuffer = malloc(bytesRead); 100 | assert(decompBuffer); 101 | 102 | /* register qatSequenceProducer */ 103 | ZSTD_registerSequenceProducer( 104 | zc, 105 | sequenceProducerState, 106 | qatSequenceProducer 107 | ); 108 | 109 | res = ZSTD_CCtx_setParameter(zc, ZSTD_c_enableSeqProducerFallback, 1); 110 | if ((int)res <= 0) { 111 | printf("Failed to set fallback\n"); 112 | goto exit; 113 | } 114 | 115 | /* compress */ 116 | cSize = ZSTD_compress2(zc, dstBuffer, dstBufferSize, srcBuffer, bytesRead); 117 | if ((int)cSize <= 0) { 118 | printf("Compress failed\n"); 119 | goto exit; 120 | } 121 | 122 | /* decompress */ 123 | res = ZSTD_decompress(decompBuffer, inputFileSize, dstBuffer, cSize); 124 | if (res != bytesRead) { 125 | printf("Decompressed size in not equal to sourece size\n"); 126 | goto exit; 127 | } 128 | 129 | /* compare original buffer with decompressed output */ 130 | if (memcmp(decompBuffer, srcBuffer, bytesRead) == 0) { 131 | printf("Compression and decompression were successful!\n"); 132 | printf("Source size: %lu\n", bytesRead); 133 | printf("Compressed size: %lu\n", cSize); 134 | } else { 135 | printf("ERROR: input and validation buffers don't match!\n"); 136 | } 137 | 138 | exit: 139 | ZSTD_freeCCtx(zc); 140 | QZSTD_freeSeqProdState(sequenceProducerState); 141 | QZSTD_stopQatDevice(); 142 | free(srcBuffer); 143 | free(dstBuffer); 144 | free(decompBuffer); 145 | return 0; 146 | } --------------------------------------------------------------------------------