├── .gitignore
├── LICENSE
├── README.md
├── asm
    └── Z80
    │   ├── DecodeBX2-hardcore.asm
    │   ├── DecodeBX2.asm
    │   ├── DecodeE1-hardcore.asm
    │   ├── DecodeE1.asm
    │   ├── DecodeE1ZX.asm
    │   ├── DecodeLZ-hardcore.asm
    │   ├── DecodeLZ.asm
    │   ├── DecodeUE2.asm
    │   └── DecodeZX2.asm
├── bin
    └── bzpack.exe
├── llvm
    └── build.bat
├── src
    ├── BitStream.cpp
    ├── BitStream.h
    ├── CommonTypes.h
    ├── Compressor.cpp
    ├── Compressor.h
    ├── Decompressor.cpp
    ├── DijkstraParser.cpp
    ├── DijkstraParser.h
    ├── Formats.h
    ├── Main.cpp
    ├── OptimalParser.cpp
    ├── OptimalParser.h
    ├── PrefixMatcher.cpp
    ├── PrefixMatcher.h
    ├── UniversalCodes.cpp
    └── UniversalCodes.h
└── vcxproj
    ├── bzpack.sln
    ├── bzpack.vcxproj
    └── bzpack.vcxproj.filters


/.gitignore:
--------------------------------------------------------------------------------
1 | /build/*
2 | /vcxproj/*.user
3 | /vcxproj/*.vs
4 | 


--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
 1 | BSD 2-Clause License
 2 | 
 3 | Copyright (c) 2021, Milos Bazelides
 4 | All rights reserved.
 5 | 
 6 | Redistribution and use in source and binary forms, with or without
 7 | modification, are permitted provided that the following conditions are met:
 8 | 
 9 | 1. Redistributions of source code must retain the above copyright notice, this
10 |    list of conditions and the following disclaimer.
11 | 
12 | 2. Redistributions in binary form must reproduce the above copyright notice,
13 |    this list of conditions and the following disclaimer in the documentation
14 |    and/or other materials provided with the distribution.
15 | 
16 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
17 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
18 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
19 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
20 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
21 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
22 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
23 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
24 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
25 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
26 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
  1 | # bzpack
  2 | 
  3 | Bzpack is a data compression utility designed for retrocomputing and sizecoding enthusiasts. Given the stringent size limits
  4 | on programs like 256-byte, 512-byte, or 1024-byte intros, implementing a compact decoder becomes as crucial as the efficiency
  5 | of the compression format. While Bzpack isn't intended to be a general-purpose packer like Einar Saukas' excellent
  6 | [ZX0](https://github.com/einar-saukas/ZX0), it aims to strike a balance between simplicity and efficiency, since advanced
  7 | coding schemes may not produce a short enough stream to justify the larger decoder size. Special consideration has been given
  8 | to vintage computing platform Sinclair ZX Spectrum.
  9 | 
 10 | ## Usage
 11 | 
 12 | Bzpack is a command-line utility with the following usage format:
 13 | 
 14 | `bzpack.exe [-lz|-e1|-e1zx|-bx2|-ue2] [-r] [-e] [-o] [-l] <inputFile> [outputFile]`
 15 | 
 16 | For example, to compress a file called "demo.bin" using the BX2 format with the end-of-stream marker, the command would be:
 17 | 
 18 | `bzpack.exe -bx2 -e demo.bin demo.bx2`
 19 | 
 20 | Here’s a list of supported command-line options (excluding compression format names):
 21 | 
 22 | * `-r` - Compress (and decompress) in reverse direction. In practice, this option helps reduce the decoder size.
 23 | * `-e` - Add an end-of-stream marker. Useful for general-purpose decompression, but often unnecessary for minimalist programs.
 24 | * `-o` - Extend the offset range by 1. Supported by some formats; can produce a slightly shorter stream at the cost of a larger
 25 | decoder.
 26 | * `-l` - Extend the block length by 1. Supported by some formats; similarly, can result in a shorter stream, but requires a
 27 | larger decoder.
 28 | 
 29 | ## Format Overview
 30 | 
 31 | All supported formats are based on the Lempel–Ziv–Storer–Szymanski algorithm. The compressed stream consists of two block types:
 32 | 
 33 | * **Literals** - Strings of uncompressed bytes stored directly in the stream.
 34 | * **Matches** - Repeated byte sequences represented as offset-length pairs, where the offset refers to already decompressed data
 35 | relative to the current output position.
 36 | 
 37 | The encoding methods for literals and matches vary between formats, and their efficiency depends on the structure of the input
 38 | data. Therefore, trying multiple formats is recommended to determine the best fit. Generally, numbers are represented either as
 39 | raw bytes or as Elias-Gamma values, read from a bit stream that works independently of natural byte boundaries.
 40 | 
 41 | ### Elias-Gamma Encoding
 42 | 
 43 | The canonical form of the Elias-Gamma code consists of **N** leading zeroes followed by a **(N + 1)**-bit binary number. For
 44 | example, the number 12 is encoded as 000**1100**. In his paper "Universal codeword sets and representations of the integers",
 45 | Peter Elias also proposed an alternative representation in which the bits are interleaved: **1**0**1**0**0**0**0**. In this
 46 | format, the most significant bit is assumed, and the zeroes act as 1-bit flags indicating whether another significant bit
 47 | follows. This representation is particularly well-suited for efficient decoder implementation in assembly language. Bzpack
 48 | adopts this approach with one minor tweak: the flags are inverted. As a result, the actual code for the number 12 becomes
 49 | 1**1**1**0**1**0**0, where:
 50 | 
 51 | * The most significant bit is not stored.
 52 | * Each subsequent significant bit is preceded by a 1, indicating its presence.
 53 | * A 0 marks the end of the sequence.
 54 | 
 55 | #### Elias-Gamma 1..N vs 2..N
 56 | 
 57 | The Elias-Gamma code represents positive integers in the range 1..N. However, by shifting the range to 2..N, the resulting
 58 | codewords can sometimes be optimized for the most common match lengths.
 59 | 
 60 | Standard Elias-Gamma code for the range 1..N:
 61 | ```
 62 | 1: 0
 63 | 2: 100
 64 | 3: 110
 65 | 4: 10100
 66 | 5: 10110
 67 | 6: 11100
 68 | 7: 11110
 69 | ```
 70 | Offset Elias-Gamma code for the range 2..N:
 71 | ```
 72 | 2: 00
 73 | 3: 10
 74 | 4: 0100
 75 | 5: 0110
 76 | 6: 1100
 77 | 7: 1110
 78 | 8: 010100
 79 | ```
 80 | In the following text, the symbol `E1` represents the Elias-Gamma code for the range 1..N, while `E2` represents the
 81 | Elias-Gamma code for 2..N.
 82 | 
 83 | ## Description of Supported Formats
 84 | 
 85 | ### LZ
 86 | 
 87 | LZ is a straightforward, byte-aligned format interpreted as follows:
 88 | 
 89 | * `ccccccc1` – Copy the next `ccccccc` bytes to the output.
 90 | * `ccccccc0`, `ffffffff` – Copy `ccccccc` bytes from an offset of `ffffffff`, relative to the current output position.
 91 | * `00000000` or `00000001` – End of stream.
 92 | 
 93 | The compression ratio is decent but not exceptional. However, the decoder is extremely compact, making it usable in highly
 94 | constrained scenarios, such as 256-byte intros. While other methods may produce a shorter compressed stream, the combined size
 95 | of the stream and decoder can make LZ the better choice.
 96 | 
 97 | Supported options:
 98 | 
 99 | * Offset increment (1..256 instead of 1..255).
100 | * Size increment (1..128 instead of 1..127).
101 | * End-of-stream marker.
102 | 
103 | ### E1
104 | 
105 | The E1 format encodes block length as an `E1` value, followed by a 1-bit flag indicating the block type:
106 | 
107 | * `E1`, `1` – Copy the next `E1` bytes to the output.
108 | * `E1`, `0`, `ffffffff` – Copy `E1 + 1` bytes from an offset of `ffffffff`, relative to the current output position.
109 | * `E1` > 255 - End of stream.
110 | 
111 | The compression ratio is significantly improved over the previous format and the decoder still manages to be short.
112 | 
113 | Supported options:
114 | 
115 | * Offset increment (1..256 instead of 1..255).
116 | * End-of-stream marker.
117 | 
118 | ### E1ZX
119 | 
120 | E1ZX is an optimized variant of E1, specifically designed for the Sinclair ZX Spectrum. While the stream length remains
121 | unchanged, certain values are stored as their complements, simplifying decompressor initialization and further reducing code
122 | size. This format is primarily intended for 512-byte and 1024-byte intros.
123 | 
124 | Supported options:
125 | 
126 | * Offset increment (1..256 instead of 1..255).
127 | 
128 | ### BX2
129 | 
130 | BX2 is a slight modification of Einar Saukas' [ZX2](https://github.com/einar-saukas/ZX2) that allows for a more efficient
131 | decoder. The format disallows consecutive literals and this implicit constraint frees up one bit of information, allowing
132 | a distinction between a regular match and a 'repeat match' that reuses the most recent offset. Blocks are encoded as follows:
133 | 
134 | * `E1`, `1` – If following a match, copy the next `E1` bytes to the output. If following a literal, copy `E1` bytes from the
135 | most recent offset.
136 | * `E1`, `0`, `ffffffff` – Copy `E1 + 1` bytes from an offset of `ffffffff`, relative to the current output position.
137 | An offset of 0 indicates the end of the stream.
138 | 
139 | The format employs an experimental exhaustive parser that, in theory, achieves a globally optimal encoding. However,
140 | compression may take even several minutes for blocks of 8 KiB or higher.
141 | 
142 | ### UE2
143 | 
144 | The UE2 format encodes literals on a per-byte basis, using 1-bit flag for each byte. If the flag is not set, it indicates
145 | a match of length `E2` combined with plain 8-bit offset.
146 | 
147 | * `1`, `bbbbbbbb` – Copy byte `bbbbbbbb` to the output.
148 | * `0`, `E2`, `ffffffff` – Copy `E2` bytes from an offset of `ffffffff`, relative to the current output position.
149 | * `E2` > 255 - End of stream.
150 | 
151 | This format tends to be hit-or-miss. It usually outperforms LZ but falls short compared to other formats. However, it can still
152 | be useful for data blocks where it happens to be a good fit.
153 | 
154 | Supported options:
155 | 
156 | * Offset increment (1..256 instead of 1..255).
157 | * End-of-stream marker.
158 | 
159 | #### Acknowledgments
160 | 
161 | I would like to acknowledge the contributions of Aleksey "introspec" Pichugin, Slavomir "Busy" Labsky and
162 | Pavel "Zilog" Cimbal. I would also like to take this opportunity to recognize the work of Einar Saukas.
163 | 


--------------------------------------------------------------------------------
/asm/Z80/DecodeBX2-hardcore.asm:
--------------------------------------------------------------------------------
 1 | ; Copyright (c) 2025, Milos "baze" Bazelides
 2 | ; This code is licensed under the BSD 2-Clause License.
 3 | 
 4 | ; Reverse BX2 "hardcore" decoder (49 bytes with setup, 44 bytes excluding setup).
 5 | 
 6 | ; This decoder is optimized for the Sinclair ZX Spectrum and operates under the following
 7 | ; assumptions, which are easily met in minimalist demoscene programs:
 8 | 
 9 | ; 1) The program is launched from BASIC using USR, with a start address of #7F80.
10 | ;    This ensures that A and C are set to 128, and B is set to 127.
11 | ; 2) The first block is a literal of at least two bytes in length.
12 | ; 3) No literal exceeds 255 bytes, and no match exceeds 254 bytes.
13 | ; 4) The compressed stream is placed immediately above the entry point.
14 | ; 5) There's no end-of-stream marker. The last block overwrites opcodes after LDDR.
15 | 
16 | 		ld	de,DestAddr
17 | 		ld	h,b
18 | 		ld	l,b
19 | 
20 | DecodeLoop	call	EliasGamma
21 | 		rla
22 | 		jr	nc,NewOffset
23 | 
24 | 		lddr
25 | 
26 | 		call	EliasGamma
27 | 		rla
28 | 		jr	c,RepOffset
29 | 
30 | NewOffset	ex	af,af'
31 | 		ld	a,(hl)
32 | 		ex	af,af'
33 | 		dec	hl
34 | 		inc	c
35 | 
36 | RepOffset	push	hl
37 | 		ex	af,af'
38 | 		ld	h,b
39 | 		ld	l,a
40 | 		ex	af,af'
41 | 		add	hl,de
42 | 		lddr
43 | 		pop	hl
44 | 		jr	DecodeLoop
45 | 
46 | EliasGamma	inc	c
47 | EliasLoop	add	a,a
48 | 		jr	nz,NoFetch
49 | 		ld	b,a
50 | 		ld	a,(hl)
51 | 		dec	hl
52 | 		rla
53 | NoFetch		ret	nc
54 | 		add	a,a
55 | 		rl	c
56 | 		jr	EliasLoop
57 | 


--------------------------------------------------------------------------------
/asm/Z80/DecodeBX2.asm:
--------------------------------------------------------------------------------
 1 | ; Copyright (c) 2025, Milos "baze" Bazelides
 2 | ; This code is licensed under the BSD 2-Clause License.
 3 | 
 4 | ; Reverse BX2 decoder (58 bytes with setup, 52 bytes excluding setup).
 5 | ; This work is inspired by Einar Saukas' ZX2 (https://github.com/einar-saukas/ZX2).
 6 | 
 7 | 		ld	hl,SrcAddr
 8 | 		ld	de,DstAddr
 9 | 
10 | 		ld	a,128
11 | DecodeLoop	call	EliasGamma
12 | 		rla
13 | 		jr	nc,NewOffset
14 | 
15 | 		lddr
16 | 
17 | 		call	EliasGamma
18 | 		rla
19 | 		jr	c,RepOffset
20 | 
21 | NewOffset	ex	af,af'
22 | 		ld	a,(hl)
23 | 		or	a
24 | 		ret	z
25 | 		ex	af,af'
26 | 		dec	hl
27 | 		inc	bc
28 | 
29 | RepOffset	push	hl
30 | 		ex	af,af'
31 | 		ld	h,0
32 | 		ld	l,a
33 | 		ex	af,af'
34 | 		add	hl,de
35 | 		lddr
36 | 		pop	hl
37 | 		jr	DecodeLoop
38 | 
39 | EliasGamma	ld	bc,1
40 | EliasLoop	add	a,a
41 | 		jr	nz,NoFetch
42 | 		ld	a,(hl)
43 | 		dec	hl
44 | 		rla
45 | NoFetch		ret	nc
46 | 		add	a,a
47 | 		rl	c
48 | 		rl	b
49 | 		jr	EliasLoop
50 | 


--------------------------------------------------------------------------------
/asm/Z80/DecodeE1-hardcore.asm:
--------------------------------------------------------------------------------
 1 | ; Copyright (c) 2021, Aleksey "introspec" Pichugin, Milos "baze" Bazelides, Pavel "Zilog" Cimbal
 2 | ; This code is licensed under the BSD 2-Clause License.
 3 | 
 4 | ; Reverse E1 "hardcore" decoder (34 bytes with setup, 29 bytes excluding setup).
 5 | 
 6 | ; This decoder is optimized for the Sinclair ZX Spectrum and operates under the following
 7 | ; assumptions:
 8 | 
 9 | ; 1) The program is launched from BASIC using USR, with a start address of #BFC0.
10 | ;    This ensures that A and C are set to %11000000, and B is set to %11000000 - 1.
11 | ; 2) The first block is a literal of at least two bytes in length.
12 | ; 3) The compressed stream is placed immediately above the entry point.
13 | ; 4) There's no end-of-stream marker. The last block overwrites opcodes after LDDR.
14 | 
15 | 		ld	de,DstAddr
16 | 		ld	h,b
17 | 		ld	l,b
18 | 
19 | EliasGamma	add	a,a
20 | 		rl	c
21 | NextBit		add	a,a
22 | 		jr	nz,NoFetch
23 | 		ld	b,a
24 | 		ld	a,(hl)
25 | 		dec	hl
26 | 		rla
27 | NoFetch		jr	c,EliasGamma
28 | 		rla
29 | 		jr	c,CopyBytes
30 | 		push	hl
31 | 		ld	l,(hl)
32 | 		ld	h,b
33 | 		add	hl,de
34 | 		inc	bc
35 | CopyBytes	lddr
36 | 		inc	c
37 | 		jr	c,NextBit
38 | 		pop	hl
39 | 		dec	hl
40 | 		jr	NextBit
41 | 


--------------------------------------------------------------------------------
/asm/Z80/DecodeE1.asm:
--------------------------------------------------------------------------------
 1 | ; Copyright (c) 2021, Aleksey "introspec" Pichugin, Milos "baze" Bazelides, Pavel "Zilog" Cimbal
 2 | ; This code is licensed under the BSD 2-Clause License.
 3 | 
 4 | ; Reverse E1 decoder (39 bytes with setup, 28 bytes excluding setup).
 5 | 
 6 | 		ld	hl,SrcAddr
 7 | 		ld	de,DstAddr
 8 | 		ld	bc,0
 9 | 		ld	a,%11000000
10 | 
11 | EliasGamma	add	a,a
12 | 		rl	c
13 | ;		ret	c		; Option to include the end-of-stream marker.
14 | NextBit		add	a,a
15 | 		jr	nz,NoFetch
16 | 		ld	a,(hl)
17 | 		dec	hl
18 | 		rla
19 | NoFetch		jr	c,EliasGamma
20 | 		rla
21 | 		jr	c,CopyBytes
22 | 		push	hl
23 | 		ld	l,(hl)
24 | 		ld	h,b
25 | 		add	hl,de
26 | ;		inc	hl		; Option to extend the offset range.
27 | 		inc	bc
28 | CopyBytes	lddr
29 | 		inc	c
30 | 		jr	c,NextBit
31 | 		pop	hl
32 | 		dec	hl
33 | 		jr	NextBit
34 | 


--------------------------------------------------------------------------------
/asm/Z80/DecodeE1ZX.asm:
--------------------------------------------------------------------------------
 1 | ; Copyright (c) 2022, Milos "baze" Bazelides
 2 | ; This code is licensed under the BSD 2-Clause License.
 3 | 
 4 | ; E1ZX decoder (usually 34..36 bytes including initialization).
 5 | 
 6 | ; The end-of-stream marker is not supported by this format. The program is expected
 7 | ; to continue after the output stream overwrites instructions just after LDDR. Both
 8 | ; decompressor versions assume aligned start address (#XX80).
 9 | 
10 | IF 0
11 | 
12 | ; On Sinclair ZX Spectrum the USR function places its argument to BC and sets A = C.
13 | ; Aligning the start address to #XX80 sets both A and C to 128 and simplifies setup.
14 | ; Normally we would need C = 0 but the most significant bit gets shifted out anyway.
15 | ; Also A = 128 is sufficient because we don't expect the second ADD A,A to set Carry.
16 | ; This is now ensured by SUB (HL). We also "sneakily" initialize B during fetch.
17 | 
18 | 		ld	hl,SrcAddr
19 | 		ld	de,DstAddr
20 | EliasGamma	add	a,a
21 | 		rl	c
22 | NextBit		add	a,a
23 | 		jr	nz,NoFetch
24 | 		ld	b,a
25 | 		sub	(hl)		; Fetch and (except for rare situations) set carry.
26 | 		dec	hl
27 | ;		scf			; Only use in case of warning.
28 | 		rla
29 | NoFetch		jr	c,EliasGamma
30 | 		rla			; Literal or phrase?
31 | 		jr	c,CopyBytes
32 | 		push	hl
33 | 		ld	l,(hl)
34 | 		ld	h,b
35 | 		add	hl,de
36 | ;		inc	hl		; Option to extend the offset range.
37 | 		inc	bc
38 | CopyBytes	lddr
39 | 		inc	c		; Prepare the most-significant Elias-Gamma bit.
40 | 		jr	c,NextBit
41 | 		pop	hl
42 | 		dec	hl
43 | 		jr	NextBit
44 | ELSE
45 | 
46 | ; If we use the USR #XX80 trick described above we can go one step further and place
47 | ; the compressed stream just above the entry point. This saves another byte because
48 | ; BC already contains the start address and HL can be initialized cheaply. However
49 | ; we must pre-decrement rather than post-decrement HL. The routine is 34 bytes long.
50 | 
51 | 		ld	h,b
52 | 		ld	l,c
53 | 		ld	de,DstAddr
54 | EliasGamma	add	a,a
55 | 		rl	c
56 | NextBit		add	a,a
57 | 		jr	nz,NoFetch
58 | 		ld	b,a
59 | 		dec	hl
60 | 		sub	(hl)		; Fetch and (except for rare situations) set carry.
61 | ;		scf			; Only use in case of warning.
62 | 		rla
63 | NoFetch		jr	c,EliasGamma
64 | 		rla			; Literal or phrase?
65 | 		jr	c,CopyBytes
66 | 		dec	hl
67 | 		push	hl
68 | 		ld	l,(hl)
69 | 		ld	h,b
70 | 		add	hl,de
71 | ;		inc	hl		; Option to extend the offset range.
72 | 		inc	bc
73 | CopyBytes	lddr
74 | 		inc	c		; Prepare the most-significant Elias-Gamma bit.
75 | 		jr	c,NextBit
76 | 		pop	hl
77 | 		jr	NextBit
78 | ENDIF
79 | 


--------------------------------------------------------------------------------
/asm/Z80/DecodeLZ-hardcore.asm:
--------------------------------------------------------------------------------
 1 | ; Copyright (c) 2022, Milos "baze" Bazelides
 2 | ; This code is licensed under the BSD 2-Clause License.
 3 | 
 4 | ; Reverse LZS "hardcore" decoder (23 bytes with setup, 18 bytes excluding setup).
 5 | 
 6 | ; This decoder is optimized for the Sinclair ZX Spectrum and operates under the following
 7 | ; assumptions:
 8 | 
 9 | ; 1) The program is launched from BASIC using USR, with a start address of #XX00,  
10 | ;    ensuring that register C is set to 0.  
11 | ; 2) The compressed stream is located immediately above the entry point.  
12 | ; 3) There's no end-of-stream marker. The last block overwrites opcodes after LDDR.  
13 | 
14 | 		ld	de,DstAddr
15 | 		push	bc
16 | 		ld	b,c
17 | DecodeLoop1	pop	hl
18 | 		dec	hl
19 | DecodeLoop2	ld	c,(hl)
20 | 		dec	hl
21 | 		srl	c
22 | 		jr	c,CopyBytes
23 | 		push	hl
24 | 		ld	l,(hl)
25 | 		ld	h,b
26 | 		add	hl,de
27 | CopyBytes	lddr
28 | 		jr	nc,DecodeLoop1
29 | 		jr	DecodeLoop2
30 | 


--------------------------------------------------------------------------------
/asm/Z80/DecodeLZ.asm:
--------------------------------------------------------------------------------
 1 | ; Copyright (c) 2017, Milos "baze" Bazelides
 2 | ; This code is licensed under the BSD 2-Clause License.
 3 | 
 4 | ; Reverse LZS decoder (26 bytes with setup, 18 bytes excluding setup).
 5 | 
 6 | ; The end-of-stream marker can be omitted if the output stream overwrites opcodes  
 7 | ; immediately after LDDR.
 8 | 
 9 | 		ld	hl,SrcAddr
10 | 		ld	de,DstAddr
11 | 		ld	b,0		; Ideally, some values should be "reused".
12 | DecodeLoop	ld	c,(hl)
13 | 		dec	hl
14 | 		srl	c
15 | ;		ret	z		; Option to include the end-of-stream marker.
16 | ;		inc	c		; Option to extend the block length.
17 | 		jr	c,CopyBytes
18 | 		push	hl
19 | 		ld	l,(hl)
20 | 		ld	h,b
21 | 		add	hl,de
22 | ;		inc	hl		; Option to extend the offset range.
23 | CopyBytes	lddr
24 | 		jr	c,DecodeLoop
25 | 		pop	hl
26 | 		dec	hl
27 | 		jr	DecodeLoop
28 | 


--------------------------------------------------------------------------------
/asm/Z80/DecodeUE2.asm:
--------------------------------------------------------------------------------
 1 | ; Copyright (c) 2021, Milos "baze" Bazelides
 2 | ; This code is licensed under the BSD 2-Clause License.
 3 | 
 4 | ; Reverse UE2 decoder (44 bytes with setup, 36 bytes excluding setup).
 5 | 
 6 | ; The end-of-stream marker can be omitted if the output stream overwrites opcodes  
 7 | ; immediately after LDDR.
 8 | 
 9 | 		ld	hl,SrcAddr
10 | 		ld	de,DstAddr
11 | 		ld	a,%10000000
12 | 
13 | DecodeLoop	ld	c,1
14 | 		call	ReadBit
15 | 		jr	c,CopyBytes
16 | 
17 | EliasGamma	call	ReadBit
18 | 		rl	c
19 | ;		ret	c		; Option to include the end-of-stream marker.
20 | 		call	ReadBit
21 | 		jr	c,EliasGamma
22 | 
23 | 		push	hl
24 | 		ld	l,(hl)
25 | 		ld	h,b
26 | 		add	hl,de
27 | ;		inc	hl		; Option to extend the offset range.
28 | CopyBytes	lddr
29 | 		jr	c,DecodeLoop
30 | 		pop	hl
31 | 		dec	hl
32 | 		jr	DecodeLoop
33 | 
34 | ReadBit		add	a,a
35 | 		ret	nz
36 | 		ld	b,a
37 | 		ld	a,(hl)
38 | 		dec	hl
39 | 		rla
40 | 		ret
41 | 


--------------------------------------------------------------------------------
/asm/Z80/DecodeZX2.asm:
--------------------------------------------------------------------------------
 1 | ; Copyright (c) 2025, Milos "baze" Bazelides
 2 | ; This code is licensed under the BSD 2-Clause License.
 3 | 
 4 | ; Experimental ZX2 decoder (61 bytes with initialization, 55 bytes excluding initialization).
 5 | ; This work is derived from Einar Saukas' ZX2 (https://github.com/einar-saukas/ZX2).
 6 | 
 7 | ; The decoder assumes reverse order.
 8 | 
 9 | 		ld	hl,SrcAddr
10 | 		ld	de,DstAddr
11 | 
12 | 		ld	a,128
13 | Literal		call	EliasGamma
14 | 		lddr
15 | 		rla
16 | 		jr	nc,NewOffset
17 | 
18 | 		call	EliasGamma
19 | RepOffset	push	hl
20 | 		ex	af,af'
21 | 		ld	h,0
22 | 		ld	l,a
23 | 		ex	af,af'
24 | 		add	hl,de
25 | 		lddr
26 | 		pop	hl
27 | 		rla
28 | 		jr	c,Literal
29 | 
30 | NewOffset	ex	af,af'
31 | 		ld	a,(hl)
32 | 		inc	a
33 | 		ret	z
34 | 		dec	hl
35 | 		ex	af,af'
36 | 		call	EliasGamma
37 | 		inc	bc
38 | 		jr	RepOffset
39 | 
40 | EliasGamma	ld	bc,1
41 | EliasLoop	add	a,a
42 | 		jr	nz,NoFetch
43 | 		ld	a,(hl)
44 | 		dec	hl
45 | 		rla
46 | NoFetch		ret	nc
47 | 		add	a,a
48 | 		rl	c
49 | 		rl	b
50 | 		jr	EliasLoop
51 | 


--------------------------------------------------------------------------------
/bin/bzpack.exe:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/mbaze/bzpack/9adc68a4a42a3b9779e0c5810e4c0e36e063bd31/bin/bzpack.exe


--------------------------------------------------------------------------------
/llvm/build.bat:
--------------------------------------------------------------------------------
1 | clang++ -std=c++14 -O3 -o ../bin/bzpack.exe ../src/*.cpp
2 | 


--------------------------------------------------------------------------------
/src/BitStream.cpp:
--------------------------------------------------------------------------------
  1 | // Copyright (c) 2021, Milos "baze" Bazelides
  2 | // This code is licensed under the BSD 2-Clause License.
  3 | 
  4 | #include <algorithm>
  5 | #include "BitStream.h"
  6 | 
  7 | size_t BitStream::Size() const
  8 | {
  9 |     return mBytes.size();
 10 | }
 11 | 
 12 | const uint8_t* BitStream::Data() const
 13 | {
 14 |     return mBytes.data();
 15 | }
 16 | 
 17 | void BitStream::Reverse()
 18 | {
 19 |     std::reverse(mBytes.begin(), mBytes.end());
 20 | }
 21 | 
 22 | void BitStream::ReadReset()
 23 | {
 24 |     mReadMask = 0;
 25 |     mReadBitPos = 0;
 26 |     mReadBytePos = 0;
 27 | }
 28 | 
 29 | void BitStream::WriteReset()
 30 | {
 31 |     mBytes.clear();
 32 |     mWriteBitNum = 0;
 33 |     mWriteBitPos = 0;
 34 |     mIssueCarryWarning = false;
 35 | }
 36 | 
 37 | void BitStream::WriteBit(bool value)
 38 | {
 39 |     if (mWriteBitNum == 0)
 40 |     {
 41 |         mWriteBitPos = mBytes.size();
 42 |         mBytes.emplace_back(0);
 43 |     }
 44 | 
 45 |     mWriteBitNum = --mWriteBitNum & 7;
 46 |     mBytes[mWriteBitPos] |= (value << mWriteBitNum);
 47 | }
 48 | 
 49 | void BitStream::WriteByte(uint8_t value)
 50 | {
 51 |     mBytes.emplace_back(value);
 52 | }
 53 | 
 54 | uint32_t BitStream::ReadBit()
 55 | {
 56 |     mReadMask >>= 1;
 57 | 
 58 |     if (mReadMask == 0)
 59 |     {
 60 |         mReadMask = 128;
 61 |         mReadBitPos = mReadBytePos++;
 62 |     }
 63 | 
 64 |     return (mBytes[mReadBitPos] & mReadMask) > 0;
 65 | }
 66 | 
 67 | uint8_t BitStream::ReadByte()
 68 | {
 69 |     return mBytes[mReadBytePos++];
 70 | }
 71 | 
 72 | // These methods are only required by the E1ZX format.
 73 | 
 74 | void BitStream::WriteBitNeg(bool value)
 75 | {
 76 |     if (mWriteBitNum == 0)
 77 |     {
 78 |         if (mBytes.size())
 79 |         {
 80 |             mBytes[mWriteBitPos] = -mBytes[mWriteBitPos];
 81 |             mIssueCarryWarning |= !mBytes[mWriteBitPos];
 82 |         }
 83 | 
 84 |         mWriteBitPos = mBytes.size();
 85 |         mBytes.emplace_back(0);
 86 |     }
 87 | 
 88 |     mWriteBitNum = --mWriteBitNum & 7;
 89 |     mBytes[mWriteBitPos] |= (value << mWriteBitNum);
 90 | }
 91 | 
 92 | uint32_t BitStream::ReadBitNeg()
 93 | {
 94 |     mReadMask >>= 1;
 95 | 
 96 |     if (mReadMask == 0)
 97 |     {
 98 |         mReadMask = 128;
 99 |         mReadBitPos = mReadBytePos++;
100 |     }
101 | 
102 |     return (-mBytes[mReadBitPos] & mReadMask) > 0;
103 | }
104 | 
105 | void BitStream::FlushBitsNeg()
106 | {
107 |     // Setting unused bits to 1 reduces the chance of there being a zero byte.
108 | 
109 |     if (mBytes.size())
110 |     {
111 |         uint8_t padding = (1 << mWriteBitNum) - 1;
112 |         mBytes[mWriteBitPos] = -(mBytes[mWriteBitPos] | padding);
113 |         mIssueCarryWarning |= !mBytes[mWriteBitPos];
114 |     }
115 | }
116 | 
117 | bool BitStream::IssueCarryWarning() const
118 | {
119 |     return mIssueCarryWarning;
120 | }
121 | 


--------------------------------------------------------------------------------
/src/BitStream.h:
--------------------------------------------------------------------------------
 1 | // Copyright (c) 2021, Milos "baze" Bazelides
 2 | // This code is licensed under the BSD 2-Clause License.
 3 | 
 4 | #ifndef BIT_STREAM_H
 5 | #define BIT_STREAM_H
 6 | 
 7 | #include <vector>
 8 | 
 9 | class BitStream
10 | {
11 | 
12 | public:
13 | 
14 |     size_t Size() const;
15 |     const uint8_t* Data() const;
16 |     void Reverse();
17 | 
18 |     void ReadReset();
19 |     void WriteReset();
20 | 
21 |     void WriteBit(bool value);
22 |     void WriteByte(uint8_t value);
23 | 
24 |     uint32_t ReadBit();
25 |     uint8_t ReadByte();
26 | 
27 |     // These methods are only required by the E1ZX format.
28 | 
29 |     void WriteBitNeg(bool value);
30 |     uint32_t ReadBitNeg();
31 |     void FlushBitsNeg();
32 |     bool IssueCarryWarning() const;
33 | 
34 | private:
35 | 
36 |     std::vector<uint8_t> mBytes;
37 | 
38 |     uint8_t mWriteBitNum = 0;
39 |     size_t mWriteBitPos = 0;
40 | 
41 |     uint8_t mReadMask = 0;
42 |     size_t mReadBitPos = 0;
43 |     size_t mReadBytePos = 0;
44 | 
45 |     bool mIssueCarryWarning = false;
46 | };
47 | 
48 | #endif // BIT_STREAM_H
49 | 


--------------------------------------------------------------------------------
/src/CommonTypes.h:
--------------------------------------------------------------------------------
 1 | // Copyright (c) 2025, Milos "baze" Bazelides
 2 | // This code is licensed under the BSD 2-Clause License.
 3 | 
 4 | #ifndef COMMON_TYPES_H
 5 | #define COMMON_TYPES_H
 6 | 
 7 | #include <cstdint>
 8 | 
 9 | struct Match
10 | {
11 |     Match(uint16_t length, uint16_t offset):
12 |         length{length}, offset{offset} {}
13 | 
14 |     uint16_t length;
15 |     uint16_t offset;
16 | };
17 | 
18 | struct ParseStep
19 | {
20 |     ParseStep(uint16_t length, uint16_t offset):
21 |         length{length}, offset{offset} {}
22 | 
23 |     uint16_t length;
24 |     uint16_t offset;
25 | };
26 | 
27 | #endif // COMMON_TYPES_H
28 | 


--------------------------------------------------------------------------------
/src/Compressor.cpp:
--------------------------------------------------------------------------------
  1 | // Copyright (c) 2021, Milos "baze" Bazelides
  2 | // This code is licensed under the BSD 2-Clause License.
  3 | 
  4 | #include <cassert>
  5 | #include <algorithm>
  6 | #include "Compressor.h"
  7 | #include "OptimalParser.h"
  8 | #include "DijkstraParser.h"
  9 | #include "UniversalCodes.h"
 10 | 
 11 | #ifdef _DEBUG
 12 | #define VERIFY
 13 | #endif // _DEBUG
 14 | 
 15 | BitStream EncodeLZ(const uint8_t* pInput, const std::vector<ParseStep>& parse, const Format& format)
 16 | {
 17 |     if (format.Id() != FormatId::LZ || !parse.size())
 18 |         return {};
 19 | 
 20 |     BitStream stream;
 21 | 
 22 |     for (const ParseStep& parseStep: parse)
 23 |     {
 24 |         uint16_t length = format.ExtendLength() ? parseStep.length - 1 : parseStep.length;
 25 | 
 26 |         if (parseStep.offset)
 27 |         {
 28 |             uint16_t offset = format.ExtendOffset() ? parseStep.offset - 1 : parseStep.offset;
 29 | 
 30 |             stream.WriteByte(static_cast<uint8_t>(length << 1));
 31 |             stream.WriteByte(static_cast<uint8_t>(offset));
 32 | 
 33 |             pInput += parseStep.length;
 34 |         }
 35 |         else
 36 |         {
 37 |             stream.WriteByte(static_cast<uint8_t>(length << 1) | 1);
 38 | 
 39 |             for (uint16_t i = 0; i < parseStep.length; i++)
 40 |             {
 41 |                 stream.WriteByte(*pInput++);
 42 |             }
 43 |         }
 44 |     }
 45 | 
 46 |     if (format.AddEndMarker())
 47 |     {
 48 |         stream.WriteByte(0);
 49 |     }
 50 | 
 51 |     return stream;
 52 | }
 53 | 
 54 | BitStream EncodeE1(const uint8_t* pInput, const std::vector<ParseStep>& parse, const Format& format)
 55 | {
 56 |     if (format.Id() != FormatId::E1 || !parse.size())
 57 |         return {};
 58 | 
 59 |     BitStream stream;
 60 | 
 61 |     for (const ParseStep& parseStep: parse)
 62 |     {
 63 |         if (parseStep.offset)
 64 |         {
 65 |             uint16_t offset = format.ExtendOffset() ? parseStep.offset - 1 : parseStep.offset;
 66 | 
 67 |             EncodeElias1(stream, parseStep.length - 1);
 68 |             stream.WriteBit(0);
 69 |             stream.WriteByte(static_cast<uint8_t>(offset));
 70 | 
 71 |             pInput += parseStep.length;
 72 |         }
 73 |         else
 74 |         {
 75 |             EncodeElias1(stream, parseStep.length);
 76 |             stream.WriteBit(1);
 77 | 
 78 |             for (uint16_t i = 0; i < parseStep.length; i++)
 79 |             {
 80 |                 stream.WriteByte(*pInput++);
 81 |             }
 82 |         }
 83 |     }
 84 | 
 85 |     if (format.AddEndMarker())
 86 |     {
 87 |         for (uint16_t i = 0; i < 16; i++)
 88 |         {
 89 |             stream.WriteBit(1);
 90 |         }
 91 | 
 92 |         stream.WriteBit(0);
 93 |     }
 94 | 
 95 |     return stream;
 96 | }
 97 | 
 98 | BitStream EncodeE1ZX(const uint8_t* pInput, const std::vector<ParseStep>& parse, const Format& format)
 99 | {
100 |     if (format.Id() != FormatId::E1ZX || !parse.size())
101 |         return {};
102 | 
103 |     BitStream stream;
104 | 
105 |     for (const ParseStep& parseStep: parse)
106 |     {
107 |         if (parseStep.offset)
108 |         {
109 |             uint16_t offset = format.ExtendOffset() ? parseStep.offset - 1 : parseStep.offset;
110 | 
111 |             EncodeElias1Neg(stream, parseStep.length - 1);
112 |             stream.WriteBitNeg(0);
113 |             stream.WriteByte(static_cast<uint8_t>(offset));
114 | 
115 |             pInput += parseStep.length;
116 |         }
117 |         else
118 |         {
119 |             EncodeElias1Neg(stream, parseStep.length);
120 |             stream.WriteBitNeg(1);
121 | 
122 |             for (size_t i = 0; i < parseStep.length; i++)
123 |             {
124 |                 stream.WriteByte(*pInput++);
125 |             }
126 |         }
127 |     }
128 | 
129 |     stream.FlushBitsNeg();
130 | 
131 |     return stream;
132 | }
133 | 
134 | BitStream EncodeBX2(const uint8_t* pInput, const std::vector<ParseStep>& parse, const Format& format)
135 | {
136 |     if (format.Id() != FormatId::BX2 || !parse.size())
137 |         return {};
138 | 
139 |     BitStream stream;
140 |     uint16_t repOffset = 0;
141 |     bool wasLiteral = false;
142 | 
143 |     for (const ParseStep& parseStep: parse)
144 |     {
145 |         if (parseStep.offset)
146 |         {
147 |             if (wasLiteral && (parseStep.offset == repOffset))
148 |             {
149 |                 EncodeElias1(stream, parseStep.length);
150 |                 stream.WriteBit(1);
151 |             }
152 |             else
153 |             {
154 |                 EncodeElias1(stream, parseStep.length - 1);
155 |                 stream.WriteBit(0);
156 |                 stream.WriteByte(static_cast<uint8_t>(parseStep.offset));
157 |             }
158 | 
159 |             pInput += parseStep.length;
160 |             repOffset = parseStep.offset;
161 |         }
162 |         else
163 |         {
164 |             EncodeElias1(stream, parseStep.length);
165 |             stream.WriteBit(1);
166 | 
167 |             for (uint16_t i = 0; i < parseStep.length; i++)
168 |             {
169 |                 stream.WriteByte(*pInput++);
170 |             }
171 |         }
172 | 
173 |         wasLiteral = !parseStep.offset;
174 |     }
175 | 
176 |     if (format.AddEndMarker())
177 |     {
178 |         EncodeElias1(stream, 1);
179 |         stream.WriteByte(0);
180 |     }
181 | 
182 |     return stream;
183 | }
184 | 
185 | BitStream EncodeUE2(const uint8_t* pInput, const std::vector<ParseStep>& parse, const Format& format)
186 | {
187 |     if (format.Id() != FormatId::UE2 || !parse.size())
188 |         return {};
189 | 
190 |     BitStream stream;
191 | 
192 |     for (const ParseStep& parseStep: parse)
193 |     {
194 |         if (parseStep.offset)
195 |         {
196 |             uint16_t offset = format.ExtendOffset() ? parseStep.offset - 1 : parseStep.offset;
197 | 
198 |             stream.WriteBit(0);
199 |             EncodeElias2(stream, parseStep.length);
200 |             stream.WriteByte(static_cast<uint8_t>(offset));
201 | 
202 |             pInput += parseStep.length;
203 |         }
204 |         else
205 |         {
206 |             for (size_t i = 0; i < parseStep.length; i++)
207 |             {
208 |                 stream.WriteBit(1);
209 |                 stream.WriteByte(*pInput++);
210 |             }
211 |         }
212 |     }
213 | 
214 |     if (format.AddEndMarker())
215 |     {
216 |         stream.WriteBit(0);
217 | 
218 |         for (size_t i = 0; i < 15; i++)
219 |         {
220 |             stream.WriteBit(1);
221 |         }
222 | 
223 |         stream.WriteBit(0);
224 |     }
225 | 
226 |     return stream;
227 | }
228 | 
229 | BitStream Compress(uint8_t* pInput, uint16_t inputSize, const Format& format)
230 | {
231 |     if (pInput == nullptr || inputSize == 0)
232 |         return {};
233 | 
234 |     if (format.Reverse())
235 |     {
236 |         std::reverse(pInput, pInput + inputSize);
237 |     }
238 | 
239 |     BitStream stream;
240 |     std::vector<ParseStep> parse;
241 | 
242 |     switch (format.Id())
243 |     {
244 |         case FormatId::LZ:
245 |             parse = Parse(pInput, inputSize, format);
246 |             stream = EncodeLZ(pInput, parse, format);
247 |             break;
248 | 
249 |         case FormatId::E1:
250 |             parse = Parse(pInput, inputSize, format);
251 |             stream = EncodeE1(pInput, parse, format);
252 |             break;
253 | 
254 |         case FormatId::E1ZX:
255 |             parse = Parse(pInput, inputSize, format);
256 |             stream = EncodeE1ZX(pInput, parse, format);
257 |             break;
258 | 
259 |         case FormatId::BX2:
260 |         {
261 |             DijkstraParser parser(pInput, inputSize, format);
262 |             parse = parser.Parse();
263 |             stream = EncodeBX2(pInput, parse, format);
264 |             break;
265 |         }
266 | 
267 |         case FormatId::UE2:
268 |             parse = Parse(pInput, inputSize, format);
269 |             stream = EncodeUE2(pInput, parse, format);
270 |             break;
271 |     }
272 | 
273 |     if (stream.Size() == 0)
274 |         return {};
275 | 
276 | #ifdef VERIFY
277 | 
278 |     std::vector<uint8_t> data = Decompress(stream, format, inputSize);
279 |     if (data.size() == 0)
280 |         return {};
281 | 
282 |     if (format.Reverse())
283 |     {
284 |         std::reverse(data.begin(), data.end());
285 |     }
286 | 
287 |     for (uint16_t i = 0; i < inputSize; i++)
288 |     {
289 |         if (pInput[i] != data.data()[i])
290 |         {
291 |             assert(0);
292 |             return {};
293 |         }
294 |     }
295 | 
296 | #endif // VERIFY
297 | 
298 |     if (format.Reverse())
299 |     {
300 |         stream.Reverse();
301 |     }
302 | 
303 |     return stream;
304 | }
305 | 


--------------------------------------------------------------------------------
/src/Compressor.h:
--------------------------------------------------------------------------------
 1 | // Copyright (c) 2021, Milos "baze" Bazelides
 2 | // This code is licensed under the BSD 2-Clause License.
 3 | 
 4 | #ifndef COMPRESSOR_H
 5 | #define COMPRESSOR_H
 6 | 
 7 | #include "BitStream.h"
 8 | #include "Formats.h"
 9 | 
10 | BitStream Compress(uint8_t* pInput, uint16_t inputSize, const Format& format);
11 | std::vector<uint8_t> Decompress(BitStream& stream, const Format& format, uint16_t inputSize);
12 | 
13 | #endif // COMPRESSOR_H
14 | 


--------------------------------------------------------------------------------
/src/Decompressor.cpp:
--------------------------------------------------------------------------------
  1 | // Copyright (c) 2021, Milos "baze" Bazelides
  2 | // This code is licensed under the BSD 2-Clause License.
  3 | 
  4 | #include "CommonTypes.h"
  5 | #include "Compressor.h"
  6 | #include "UniversalCodes.h"
  7 | 
  8 | std::vector<uint8_t> DecodeLZ(BitStream& stream, const Format& format, uint16_t inputSize)
  9 | {
 10 |     if (format.Id() != FormatId::LZ)
 11 |         return {};
 12 | 
 13 |     std::vector<uint8_t> data;
 14 |     stream.ReadReset();
 15 | 
 16 |     while (true)
 17 |     {
 18 |         uint16_t length = stream.ReadByte();
 19 | 
 20 |         if (format.AddEndMarker() && length == 0)
 21 |             break;
 22 | 
 23 |         bool isLiteral = (length & 1);
 24 |         length >>= 1;
 25 | 
 26 |         if (format.ExtendLength())
 27 |         {
 28 |             length++;
 29 |         }
 30 | 
 31 |         if (isLiteral)
 32 |         {
 33 |             while (length--)
 34 |             {
 35 |                 data.emplace_back(stream.ReadByte());
 36 |             }
 37 |         }
 38 |         else
 39 |         {
 40 |             uint16_t offset = stream.ReadByte();
 41 | 
 42 |             if (format.ExtendOffset())
 43 |             {
 44 |                 offset++;
 45 |             }
 46 | 
 47 |             while (length--)
 48 |             {
 49 |                 data.emplace_back(data[data.size() - offset]);
 50 |             }
 51 |         }
 52 | 
 53 |         if (!format.AddEndMarker() && data.size() >= inputSize)
 54 |             break;
 55 |     }
 56 | 
 57 |     return data;
 58 | }
 59 | 
 60 | std::vector<uint8_t> DecodeE1(BitStream& stream, const Format& format, uint16_t inputSize)
 61 | {
 62 |     if (format.Id() != FormatId::E1)
 63 |         return {};
 64 | 
 65 |     std::vector<uint8_t> data;
 66 |     stream.ReadReset();
 67 | 
 68 |     while (true)
 69 |     {
 70 |         uint16_t length = DecodeElias1(stream);
 71 | 
 72 |         if (format.AddEndMarker() && length > 255)
 73 |             break;
 74 | 
 75 |         if (stream.ReadBit())
 76 |         {
 77 |             while (length--)
 78 |             {
 79 |                 data.emplace_back(stream.ReadByte());
 80 |             }
 81 |         }
 82 |         else
 83 |         {
 84 |             length++;
 85 |             size_t offset = stream.ReadByte();
 86 | 
 87 |             if (format.ExtendOffset())
 88 |             {
 89 |                 offset++;
 90 |             }
 91 | 
 92 |             while (length--)
 93 |             {
 94 |                 data.emplace_back(data[data.size() - offset]);
 95 |             }
 96 |         }
 97 | 
 98 |         if (!format.AddEndMarker() && data.size() >= inputSize)
 99 |             break;
100 |     }
101 | 
102 |     return data;
103 | }
104 | 
105 | std::vector<uint8_t> DecodeE1ZX(BitStream& stream, const Format& format, uint16_t inputSize)
106 | {
107 |     if (format.Id() != FormatId::E1ZX)
108 |         return {};
109 | 
110 |     std::vector<uint8_t> data;
111 |     stream.ReadReset();
112 | 
113 |     while (true)
114 |     {
115 |         uint16_t length = DecodeElias1Neg(stream);
116 | 
117 |         if (stream.ReadBitNeg())
118 |         {
119 |             while (length--)
120 |             {
121 |                 data.emplace_back(stream.ReadByte());
122 |             }
123 |         }
124 |         else
125 |         {
126 |             length++;
127 |             uint16_t offset = stream.ReadByte();
128 | 
129 |             if (format.ExtendOffset())
130 |             {
131 |                 offset++;
132 |             }
133 | 
134 |             while (length--)
135 |             {
136 |                 data.emplace_back(data[data.size() - offset]);
137 |             }
138 |         }
139 | 
140 |         if (data.size() >= inputSize)
141 |             break;
142 |     }
143 | 
144 |     return data;
145 | }
146 | 
147 | std::vector<uint8_t> DecodeBX2(BitStream& stream, const Format& format, uint16_t inputSize)
148 | {
149 |     if (format.Id() != FormatId::BX2)
150 |         return {};
151 | 
152 |     std::vector<uint8_t> data;
153 |     stream.ReadReset();
154 | 
155 |     uint16_t repOffset = 0;
156 |     bool wasLiteral = false;
157 | 
158 |     while (true)
159 |     {
160 |         uint16_t length = DecodeElias1(stream);
161 | 
162 |         if (stream.ReadBit())
163 |         {
164 |             if (wasLiteral)
165 |             {
166 |                 while (length--)
167 |                 {
168 |                     data.emplace_back(data[data.size() - repOffset]);
169 |                 }
170 | 
171 |                 wasLiteral = false;
172 |             }
173 |             else
174 |             {
175 |                 while (length--)
176 |                 {
177 |                     data.emplace_back(stream.ReadByte());
178 |                 }
179 | 
180 |                 wasLiteral = true;
181 |             }
182 |         }
183 |         else
184 |         {
185 |             length++;
186 |             uint16_t offset = stream.ReadByte();
187 | 
188 |             if (format.AddEndMarker() && offset == 0)
189 |                 break;
190 | 
191 |             while (length--)
192 |             {
193 |                 data.emplace_back(data[data.size() - offset]);
194 |             }
195 | 
196 |             repOffset = offset;
197 |             wasLiteral = false;
198 |         }
199 | 
200 |         if (!format.AddEndMarker() && data.size() >= inputSize)
201 |             break;
202 |     }
203 | 
204 |     return data;
205 | }
206 | 
207 | std::vector<uint8_t> DecodeUE2(BitStream& stream, const Format& format, size_t inputSize)
208 | {
209 |     if (format.Id() != FormatId::UE2)
210 |         return {};
211 | 
212 |     stream.ReadReset();
213 |     std::vector<uint8_t> data;
214 | 
215 |     while (true)
216 |     {
217 |         if (stream.ReadBit())
218 |         {
219 |             data.emplace_back(stream.ReadByte());
220 |         }
221 |         else
222 |         {
223 |             uint16_t length = DecodeElias2(stream);
224 | 
225 |             if (format.AddEndMarker() && length > 255)
226 |                 break;
227 | 
228 |             uint16_t offset = stream.ReadByte();
229 | 
230 |             if (format.ExtendOffset())
231 |             {
232 |                 offset++;
233 |             }
234 | 
235 |             while (length--)
236 |             {
237 |                 data.emplace_back(data[data.size() - offset]);
238 |             }
239 |         }
240 | 
241 |         if (!format.AddEndMarker() && data.size() >= inputSize)
242 |             break;
243 |     }
244 | 
245 |     return data;
246 | }
247 | 
248 | std::vector<uint8_t> Decompress(BitStream& stream, const Format& format, uint16_t inputSize)
249 | {
250 |     std::vector<uint8_t> data;
251 | 
252 |     switch (format.Id())
253 |     {
254 |         case FormatId::LZ:
255 |             data = DecodeLZ(stream, format, inputSize);
256 |             break;
257 | 
258 |         case FormatId::E1:
259 |             data = DecodeE1(stream, format, inputSize);
260 |             break;
261 | 
262 |         case FormatId::E1ZX:
263 |             data = DecodeE1ZX(stream, format, inputSize);
264 |             break;
265 | 
266 |         case FormatId::BX2:
267 |             data = DecodeBX2(stream, format, inputSize);
268 |             break;
269 | 
270 |         case FormatId::UE2:
271 |             data = DecodeUE2(stream, format, inputSize);
272 |             break;
273 |     }
274 | 
275 |     if (format.Reverse())
276 |     {
277 |         std::reverse(data.begin(), data.end());
278 |     }
279 | 
280 |     return data;
281 | }
282 | 


--------------------------------------------------------------------------------
/src/DijkstraParser.cpp:
--------------------------------------------------------------------------------
  1 | // Copyright (c) 2025, Milos "baze" Bazelides
  2 | // This code is licensed under the BSD 2-Clause License.
  3 | 
  4 | #include <queue>
  5 | #include "DijkstraParser.h"
  6 | 
  7 | #define COST_INDEX_64(cost, index) \
  8 |     ((static_cast<uint64_t>(cost) << 32) | static_cast<uint64_t>(index))
  9 | 
 10 | std::vector<ParseStep> DijkstraParser::Parse()
 11 | {
 12 |     uint32_t bestPathIndex = 0;
 13 | 
 14 | #ifdef BASELINE_COST_PRUNING
 15 |     uint32_t baselineCost = ComputeGreedyParseCost();
 16 | #endif // BASELINE_COST_PRUNING
 17 | 
 18 |     std::vector<uint32_t> posCosts(mInputSize, 0xFFFFFFFF);
 19 | 
 20 |     std::vector<PathNode> nodes;
 21 |     nodes.emplace_back(0, 0, 0, 0xFFFFFFFF, 0);
 22 | 
 23 |     std::priority_queue<uint64_t, std::vector<uint64_t>, std::greater<uint64_t>> queue;
 24 |     queue.emplace(0);
 25 | 
 26 |     while (!queue.empty())
 27 |     {
 28 |         uint64_t queueTop = queue.top();
 29 |         uint32_t nodeIndex = static_cast<uint32_t>(queueTop);
 30 |         uint32_t cost = static_cast<uint32_t>(queueTop >> 32);
 31 |         queue.pop();
 32 | 
 33 |         PathNode node = nodes[nodeIndex];
 34 | 
 35 |         if (node.inputPos >= mInputSize)
 36 |         {
 37 |             // End of input data reached, exit loop.
 38 |             bestPathIndex = nodeIndex;
 39 |             break;
 40 |         }
 41 | 
 42 | #ifdef BASELINE_COST_PRUNING
 43 |         // Prune this path if it can't improve upon the greedy parse.
 44 |         if (cost >= baselineCost)
 45 |             continue;
 46 | #endif // BASELINE_COST_PRUNING
 47 | 
 48 |         // Consider all available matches (if not already explored by a previous path).
 49 | 
 50 |         if (cost < posCosts[node.inputPos])
 51 |         {
 52 |             posCosts[node.inputPos] = cost;
 53 | 
 54 |             for (const Match& match: mMatcher.FindMatches(node.inputPos))
 55 |             {
 56 |                 uint32_t newPos = node.inputPos + match.length;
 57 |                 uint32_t newCost = cost + mFormat.GetMatchCost(match.length, match.offset);
 58 | 
 59 |                 if (ShouldEnqueue(newPos, match.offset, newCost))
 60 |                 {
 61 |                     queue.emplace(COST_INDEX_64(newCost, nodes.size()));
 62 |                     nodes.emplace_back(nodeIndex, newPos, match.length, match.offset, match.offset);
 63 |                 }
 64 |             }
 65 |         }
 66 | 
 67 |         // Consider all repeat matches (only after a literal).
 68 | 
 69 |         if (node.offset == 0)
 70 |         {
 71 |             for (const Match& match: mMatcher.FindMatches(node.inputPos, true))
 72 |             {
 73 |                 if (match.offset != node.repOffset)
 74 |                     continue;
 75 | 
 76 |                 uint32_t newPos = node.inputPos + match.length;
 77 |                 uint32_t newCost = cost + mFormat.GetRepMatchCost(match.length);
 78 | 
 79 |                 if (ShouldEnqueue(newPos, node.repOffset, newCost))
 80 |                 {
 81 |                     queue.emplace(COST_INDEX_64(newCost, nodes.size()));
 82 |                     nodes.emplace_back(nodeIndex, newPos, match.length, match.offset, match.offset);
 83 |                 }
 84 |             }
 85 |         }
 86 | 
 87 |         // Consider all available literals (only after a match).
 88 | 
 89 |         if (node.offset)
 90 |         {
 91 |             uint16_t maxLength = std::min<uint16_t>(mInputSize - node.inputPos, mFormat.MaxLiteralLength());
 92 | 
 93 |             for (uint16_t length = 1; length <= maxLength; length++)
 94 |             {
 95 |                 uint32_t newPos = node.inputPos + length;
 96 |                 uint32_t newCost = cost + mFormat.GetLiteralCost(length);
 97 | 
 98 |                 if (ShouldEnqueue(newPos, node.repOffset, newCost))
 99 |                 {
100 |                     queue.emplace(COST_INDEX_64(newCost, nodes.size()));
101 |                     nodes.emplace_back(nodeIndex, newPos, length, 0, node.repOffset);
102 |                 }
103 |             }
104 |         }
105 |     }
106 | 
107 |     // Backtrack and reconstruct the optimal parse sequence.
108 | 
109 |     std::vector<ParseStep> parse;
110 | 
111 |     while (bestPathIndex)
112 |     {
113 |         parse.emplace_back(nodes[bestPathIndex].length, nodes[bestPathIndex].offset);
114 |         bestPathIndex = nodes[bestPathIndex].parent;
115 |     }
116 | 
117 |     std::reverse(parse.begin(), parse.end());
118 | 
119 |     return parse;
120 | }
121 | 
122 | bool DijkstraParser::ShouldEnqueue(uint16_t inputPos, uint16_t repOffset, uint32_t cost)
123 | {
124 |     uint32_t key = (repOffset << 16) | inputPos;
125 | 
126 |     auto result = mPosRepCosts.try_emplace(key, cost);
127 |     if (result.second)
128 |         return true;
129 | 
130 |     // Prune if a new arrival to the same state doesn't reduce the cost.
131 | 
132 |     if (cost >= result.first->second)
133 |         return false;
134 | 
135 |     result.first->second = cost;
136 | 
137 |     return true;
138 | }
139 | 
140 | #ifdef BASELINE_COST_PRUNING
141 | 
142 | uint32_t DijkstraParser::ComputeGreedyParseCost()
143 | {
144 |     uint32_t cost = 0;
145 |     uint16_t inputPos = 0;
146 |     uint16_t literalLength = 0;
147 | 
148 |     while (inputPos < mInputSize)
149 |     {
150 |         Match match = mMatcher.FindLongestMatch(inputPos);
151 |         if (match.offset)
152 |         {
153 |             if (literalLength)
154 |             {
155 |                 cost += mFormat.GetLiteralCost(literalLength);
156 |                 literalLength = 0;
157 |             }
158 | 
159 |             cost += mFormat.GetMatchCost(match.length, match.offset);
160 |             inputPos += match.length;
161 |         }
162 |         else
163 |         {
164 |             literalLength++;
165 |             inputPos++;        
166 |         }
167 |     }
168 | 
169 |     if (literalLength)
170 |     {
171 |         cost += mFormat.GetLiteralCost(literalLength);
172 |     }
173 | 
174 |     return cost;
175 | }
176 | 
177 | #endif // BASELINE_COST_PRUNING
178 | 


--------------------------------------------------------------------------------
/src/DijkstraParser.h:
--------------------------------------------------------------------------------
 1 | // Copyright (c) 2025, Milos "baze" Bazelides
 2 | // This code is licensed under the BSD 2-Clause License.
 3 | 
 4 | #ifndef DIJKSTRA_PARSER_H
 5 | #define DIJKSTRA_PARSER_H
 6 | 
 7 | #include <unordered_map>
 8 | #include "Formats.h"
 9 | #include "PrefixMatcher.h"
10 | 
11 | class DijkstraParser
12 | {
13 | public:
14 | 
15 |     DijkstraParser() = delete;
16 | 
17 |     DijkstraParser(const uint8_t* pInput, uint16_t inputSize, const Format& format):
18 |         mInputPtr{pInput},
19 |         mInputSize{inputSize},
20 |         mFormat{format},
21 |         mMatcher{
22 |             pInput,
23 |             inputSize,
24 |             format.MinMatchLength(),
25 |             format.MaxMatchLength(),
26 |             format.MaxMatchOffset()
27 |         }
28 |     {}
29 | 
30 |     std::vector<ParseStep> Parse();
31 | 
32 | private:
33 | 
34 |     bool ShouldEnqueue(uint16_t inputPos, uint16_t repOffset, uint32_t cost);
35 | 
36 | #ifdef BASELINE_COST_PRUNING
37 |     uint32_t ComputeGreedyParseCost();
38 | #endif // BASELINE_COST_PRUNING
39 | 
40 |     const uint8_t* mInputPtr;
41 |     const uint16_t mInputSize;
42 |     const Format& mFormat;
43 | 
44 |     PrefixMatcher mMatcher;
45 |     std::unordered_map<uint32_t, uint32_t> mPosRepCosts;
46 | 
47 |     struct PathNode
48 |     {
49 |         PathNode(uint32_t parent, uint16_t inputPos, uint16_t length, uint16_t offset, uint16_t repOffset):
50 |             parent{parent}, inputPos{inputPos}, length{length}, offset{offset}, repOffset{repOffset} {}
51 | 
52 |         uint32_t parent;
53 |         uint16_t inputPos;
54 |         uint16_t length;
55 |         uint16_t offset;
56 |         uint16_t repOffset;
57 |     };
58 | };
59 | 
60 | #endif // DIJKSTRA_PARSER_H
61 | 


--------------------------------------------------------------------------------
/src/Formats.h:
--------------------------------------------------------------------------------
  1 | // Copyright (c) 2024, Milos "baze" Bazelides
  2 | // This code is licensed under the BSD 2-Clause License.
  3 | 
  4 | #ifndef FORMATS_H
  5 | #define FORMATS_H
  6 | 
  7 | #include "UniversalCodes.h"
  8 | 
  9 | enum FormatId
 10 | {
 11 |     LZ,
 12 |     E1,
 13 |     E1ZX,
 14 |     BX2,
 15 |     UE2,
 16 | };
 17 | 
 18 | struct FormatOptions
 19 | {
 20 |     uint8_t id: 4;
 21 |     uint8_t reverse: 1;
 22 |     uint8_t addEndMarker: 1;
 23 |     uint8_t extendOffset: 1;
 24 |     uint8_t extendLength: 1;
 25 | };
 26 | 
 27 | class Format
 28 | {
 29 | public:
 30 | 
 31 |     Format() = delete;
 32 | 
 33 |     Format(FormatOptions options):
 34 |         mFormatId(static_cast<FormatId>(options.id)),
 35 |         mReverse(options.reverse),
 36 |         mAddEndMarker(options.addEndMarker),
 37 |         mExtendOffset(options.extendOffset),
 38 |         mExtendLength(options.extendLength)
 39 |     {}
 40 | 
 41 |     virtual ~Format() = default;
 42 |     virtual uint32_t GetLiteralCost(uint16_t length) const = 0;
 43 |     virtual uint32_t GetMatchCost(uint16_t length, uint16_t offset) const = 0;
 44 |     virtual uint32_t GetRepMatchCost(uint16_t length) const = 0;
 45 | 
 46 |     FormatId Id() const { return mFormatId; }
 47 | 
 48 |     uint16_t MaxLiteralLength() const { return mMaxLiteralLength; }
 49 |     uint16_t MinMatchLength() const { return mMinMatchLength; }
 50 |     uint16_t MaxMatchLength() const { return mMaxMatchLength; }
 51 |     uint16_t MaxMatchOffset() const { return mMaxMatchOffset; }
 52 | 
 53 |     bool Reverse() const { return mReverse; }
 54 |     bool AddEndMarker() const { return mAddEndMarker; }
 55 |     bool ExtendOffset() const { return mExtendOffset; }
 56 |     bool ExtendLength() const { return mExtendLength; }
 57 | 
 58 | protected:
 59 | 
 60 |     FormatId mFormatId;
 61 | 
 62 |     // Format limits.
 63 | 
 64 |     uint16_t mMaxLiteralLength = 0xFFFF;
 65 |     uint16_t mMinMatchLength = 2;
 66 |     uint16_t mMaxMatchLength = 0xFFFF;
 67 |     uint16_t mMaxMatchOffset = 0xFFFF;
 68 | 
 69 |     // Encoding options.
 70 | 
 71 |     bool mReverse;
 72 |     bool mAddEndMarker;
 73 |     bool mExtendOffset;
 74 |     bool mExtendLength;
 75 | };
 76 | 
 77 | class FormatLZ: public Format
 78 | {
 79 | public:
 80 | 
 81 |     FormatLZ() = delete;
 82 | 
 83 |     FormatLZ(FormatOptions options): Format{options}
 84 |     {
 85 |         mMaxLiteralLength = options.extendLength ? 128 : 127;
 86 |         mMinMatchLength = 2;
 87 |         mMaxMatchLength = options.extendLength ? 128 : 127;
 88 |         mMaxMatchOffset = options.extendOffset ? 256 : 255;
 89 |     }
 90 | 
 91 |     uint32_t GetLiteralCost(uint16_t length) const override
 92 |     {
 93 |         return 8 + (length << 3);
 94 |     }
 95 | 
 96 |     uint32_t GetMatchCost(uint16_t length, uint16_t offset) const override
 97 |     {
 98 |         return 8 + 8;
 99 |     }
100 | 
101 |     uint32_t GetRepMatchCost(uint16_t length) const override
102 |     {
103 |         return 0xFFFFFFFF;
104 |     }
105 | };
106 | 
107 | class FormatE1: public Format
108 | {
109 | public:
110 | 
111 |     FormatE1() = delete;
112 | 
113 |     FormatE1(FormatOptions options): Format{options}
114 |     {
115 |         mMaxLiteralLength = 255;
116 |         mMinMatchLength = 2;
117 |         mMaxMatchLength = 256;
118 |         mMaxMatchOffset = options.extendOffset ? 256 : 255;
119 |     }
120 | 
121 |     uint32_t GetLiteralCost(uint16_t length) const override
122 |     {
123 |         return GetElias1Cost(length) + 1 + (length << 3);
124 |     }
125 | 
126 |     uint32_t GetMatchCost(uint16_t length, uint16_t offset) const override
127 |     {
128 |         return GetElias1Cost(length - 1) + 1 + 8;
129 |     }
130 | 
131 |     uint32_t GetRepMatchCost(uint16_t length) const override
132 |     {
133 |         return 0xFFFFFFFF;
134 |     }
135 | };
136 | 
137 | class FormatE1ZX: public Format
138 | {
139 | public:
140 | 
141 |     FormatE1ZX() = delete;
142 | 
143 |     FormatE1ZX(FormatOptions options): Format{options}
144 |     {
145 |         mMaxLiteralLength = 255;
146 |         mMinMatchLength = 2;
147 |         mMaxMatchLength = 256;
148 |         mMaxMatchOffset = options.extendOffset ? 256 : 255;
149 |     }
150 | 
151 |     uint32_t GetLiteralCost(uint16_t length) const override
152 |     {
153 |         return GetElias1Cost(length) + 1 + (length << 3);
154 |     }
155 | 
156 |     uint32_t GetMatchCost(uint16_t length, uint16_t offset) const override
157 |     {
158 |         return GetElias1Cost(length - 1) + 1 + 8;
159 |     }
160 | 
161 |     uint32_t GetRepMatchCost(uint16_t length) const override
162 |     {
163 |         return 0xFFFFFFFF;
164 |     }
165 | };
166 | 
167 | class FormatBX2: public Format
168 | {
169 | public:
170 | 
171 |     FormatBX2() = delete;
172 | 
173 |     FormatBX2(FormatOptions options): Format{options}
174 |     {
175 |         mMaxLiteralLength = 0xFFFF;
176 |         mMinMatchLength = 2;
177 |         mMaxMatchLength = 0xFFFF;
178 |         mMaxMatchOffset = 255;
179 |     }
180 | 
181 |     uint32_t GetLiteralCost(uint16_t length) const override
182 |     {
183 |         return GetElias1Cost(length) + 1 + (length << 3);
184 |     }
185 | 
186 |     uint32_t GetMatchCost(uint16_t length, uint16_t offset) const override
187 |     {
188 |         return GetElias1Cost(length - 1) + 1 + 8;
189 |     }
190 | 
191 |     uint32_t GetRepMatchCost(uint16_t length) const override
192 |     {
193 |         return GetElias1Cost(length) + 1;
194 |     }
195 | };
196 | 
197 | class FormatUE2: public Format
198 | {
199 | public:
200 | 
201 |     FormatUE2() = delete;
202 | 
203 |     FormatUE2(FormatOptions options): Format{options}
204 |     {
205 |         mMaxLiteralLength = 0xFFFF;
206 |         mMinMatchLength = 2;
207 |         mMaxMatchLength = 255;
208 |         mMaxMatchOffset = options.extendOffset ? 256 : 255;
209 |     }
210 | 
211 |     uint32_t GetLiteralCost(uint16_t length) const override
212 |     {
213 |         return (1 + 8) * length;
214 |     }
215 | 
216 |     uint32_t GetMatchCost(uint16_t length, uint16_t offset) const override
217 |     {
218 |         return 1 + GetElias2Cost(length) + 8;
219 |     }
220 | 
221 |     uint32_t GetRepMatchCost(uint16_t length) const override
222 |     {
223 |         return 0xFFFFFFFF;
224 |     }
225 | };
226 | 
227 | #endif // FORMATS_H
228 | 


--------------------------------------------------------------------------------
/src/Main.cpp:
--------------------------------------------------------------------------------
  1 | // Copyright (c) 2021, Milos "baze" Bazelides
  2 | // This code is licensed under the BSD 2-Clause License.
  3 | 
  4 | #include <memory>
  5 | #include <cstdio>
  6 | #include <fstream>
  7 | #include <functional>
  8 | #include <unordered_map>
  9 | #include "Compressor.h"
 10 | 
 11 | enum ErrorId
 12 | {
 13 |     InvalidParam,
 14 |     InputFileMissing,
 15 |     InputFileError,
 16 |     OutputFileError,
 17 |     FileEmpty,
 18 |     FileTooBig,
 19 |     CompressionFailed
 20 | };
 21 | 
 22 | enum WarningId
 23 | {
 24 |     ExtendLength,
 25 |     AddEndMarker,
 26 |     NoSizeGain,
 27 |     ZxExplicitCarry
 28 | };
 29 | 
 30 | void PrintError(ErrorId error, const char* pString = nullptr)
 31 | {
 32 |     printf("Error: ");
 33 | 
 34 |     switch (error)
 35 |     {
 36 |         case ErrorId::InvalidParam:
 37 |             printf("Invalid parameter %s.\n", pString);
 38 |             break;
 39 | 
 40 |         case ErrorId::InputFileMissing:
 41 |             printf("Input file is missing.\n");
 42 |             break;
 43 | 
 44 |         case ErrorId::InputFileError:
 45 |             printf("Unable to open the input file.\n");
 46 |             break;
 47 | 
 48 |         case ErrorId::OutputFileError:
 49 |             printf("Unable to create the output file.\n");
 50 |             break;
 51 | 
 52 |         case ErrorId::FileEmpty:
 53 |             printf("The input file is empty.\n");
 54 |             break;
 55 | 
 56 |         case ErrorId::FileTooBig:
 57 |             printf("The input file is too large.\n");
 58 |             break;
 59 | 
 60 |         case ErrorId::CompressionFailed:
 61 |             printf("Compression failed.\n");
 62 |             break;
 63 |     }
 64 | }
 65 | 
 66 | void PrintWarning(WarningId warning)
 67 | {
 68 |     printf("Warning: ");
 69 | 
 70 |     switch (warning)
 71 |     {
 72 |         case WarningId::ExtendLength:
 73 |             printf("Option -l is not supported by this format.\n");
 74 |             break;
 75 | 
 76 |         case WarningId::AddEndMarker:
 77 |             printf("Option -e is not supported by this format.\n");
 78 |             break;
 79 | 
 80 |         case WarningId::NoSizeGain:
 81 |             printf("No size gain after compression.\n");
 82 |             break;
 83 | 
 84 |         case WarningId::ZxExplicitCarry:
 85 |             printf("Make sure the decompressor sets Carry during fetch.\n");
 86 |             break;
 87 |     }
 88 | }
 89 | 
 90 | void CheckOptions(FormatOptions options)
 91 | {
 92 |     bool noExtLength = (options.id == FormatId::E1);
 93 |     noExtLength |= (options.id == FormatId::E1ZX);
 94 |     noExtLength |= (options.id == FormatId::UE2);
 95 | 
 96 |     if (options.extendLength && noExtLength)
 97 |     {
 98 |         PrintWarning(WarningId::ExtendLength);
 99 |     }
100 | 
101 |     if (options.id == FormatId::E1ZX && options.addEndMarker)
102 |     {
103 |         PrintWarning(WarningId::AddEndMarker);
104 |     }
105 | }
106 | 
107 | std::vector<uint8_t> ReadFile(const char* pFileName)
108 | {
109 |     std::basic_ifstream<uint8_t> file(pFileName, std::ios::binary | std::ios::ate);
110 | 
111 |     if (!file)
112 |     {
113 |         PrintError(ErrorId::InputFileError);
114 |         return {};
115 |     }
116 | 
117 |     std::streampos fileSize = file.tellg();
118 | 
119 |     if (fileSize < 0)
120 |     {
121 |         PrintError(ErrorId::InputFileError);
122 |         return {};
123 |     }
124 | 
125 |     if (fileSize == 0)
126 |     {
127 |         PrintError(ErrorId::FileEmpty);
128 |         return {};
129 |     }
130 | 
131 |     if (fileSize > 0xFFFF)
132 |     {
133 |         PrintError(ErrorId::FileTooBig);
134 |         return {};
135 |     }
136 | 
137 |     if (!file.seekg(0, std::ios_base::beg))
138 |     {
139 |         PrintError(ErrorId::InputFileError);
140 |         return {};
141 |     }
142 | 
143 |     std::vector<uint8_t> bytes(static_cast<size_t>(fileSize));
144 | 
145 |     if (!file.read(bytes.data(), fileSize))
146 |     {
147 |         PrintError(ErrorId::InputFileError);
148 |         return {};
149 |     }
150 | 
151 |     if (file.gcount() != fileSize)
152 |     {
153 |         PrintError(ErrorId::InputFileError);
154 |         return {};
155 |     }
156 | 
157 |     return bytes;
158 | }
159 | 
160 | bool WriteFile(const char* pFileName, const uint8_t* pData, size_t size)
161 | {
162 |     std::basic_ofstream<uint8_t> outputFile(pFileName, std::ios::binary);
163 | 
164 |     if (!outputFile)
165 |     {
166 |         PrintError(ErrorId::OutputFileError);
167 |         return false;
168 |     }
169 | 
170 |     if (!outputFile.write(pData, size))
171 |     {
172 |         PrintError(ErrorId::OutputFileError);
173 |         return false;
174 |     }
175 | 
176 |     return true;
177 | }
178 | 
179 | int main(int argCount, char** args)
180 | {
181 |     if (argCount < 2)
182 |     {
183 |         printf("\nUsage: bzpack.exe [-lz|-e1|-e1zx|-bx2|-ue2] [-r] [-e] [-o] [-l] <inputFile> [outputFile]\n");
184 |         printf("\nOptions:\n\n");
185 |         printf("-lz: Byte-aligned LZSS. Raw 7-bit length, raw 8-bit offset (default).\n");
186 |         printf("-e1: Elias 1..N length, raw 8-bit offset.\n");
187 |         printf("-e1zx: A version of -e1 optimized for Sinclair ZX Spectrum.\n");
188 |         printf("-bx2: Elias 1..N length, raw 8-bit offset or repeat offset.\n");
189 |         printf("-ue2: Unary literal length, Elias 2..N match length, raw 8-bit offset.\n");
190 |         printf("-r: Compress in reverse order.\n");
191 |         printf("-e: Add end-of-stream marker.\n");
192 |         printf("-o: Extend the offset range by 1.\n");
193 |         printf("-l: Extend the block length by 1.\n");
194 |         return 0;
195 |     }
196 | 
197 |     std::string suffix = ".lz";
198 |     FormatOptions options = {FormatId::LZ, 0, 0, 0, 0};
199 | 
200 |     static const std::unordered_map<std::string, std::function<void()>> actions =
201 |     {
202 |         {"-lz",   [&]() { options.id = FormatId::LZ; suffix = ".lz"; }},
203 |         {"-e1",   [&]() { options.id = FormatId::E1; suffix = ".e1"; }},
204 |         {"-e1zx", [&]() { options.id = FormatId::E1ZX; suffix = ".e1zx"; }},
205 |         {"-bx2",  [&]() { options.id = FormatId::BX2; suffix = ".bx2"; }},
206 |         {"-ue2",  [&]() { options.id = FormatId::UE2; suffix = ".ue2"; }},
207 |         {"-r",    [&]() { options.reverse = 1; }},
208 |         {"-e",    [&]() { options.addEndMarker = 1; }},
209 |         {"-o",    [&]() { options.extendOffset = 1; }},
210 |         {"-l",    [&]() { options.extendLength = 1; }}
211 |     };
212 | 
213 |     // Process command line arguments.
214 | 
215 |     std::string inputName, outputName;
216 | 
217 |     for (int i = 1; i < argCount; i++)
218 |     {
219 |         if (inputName.empty())
220 |         {
221 |             if (args[i][0] == '-')
222 |             {
223 |                 auto iAction = actions.find(args[i]);
224 |                 if (iAction != actions.end())
225 |                 {
226 |                     iAction->second();
227 |                 }
228 |                 else
229 |                 {
230 |                     PrintError(ErrorId::InvalidParam, args[i]);
231 |                     return 1;
232 |                 }
233 |             }
234 |             else
235 |             {
236 |                 inputName = args[i];
237 |             }
238 |         }
239 |         else if (outputName.empty())
240 |         {
241 |             outputName = args[i];
242 |         }
243 |         else
244 |         {
245 |             PrintError(ErrorId::InvalidParam, args[i]);
246 |             return 1;
247 |         }
248 |     }
249 | 
250 |     if (inputName.empty())
251 |     {
252 |         PrintError(ErrorId::InputFileMissing);
253 |         return 1;
254 |     }
255 | 
256 |     if (outputName.empty())
257 |     {
258 |         outputName = inputName + suffix;
259 |     }
260 | 
261 |     CheckOptions(options);
262 | 
263 |     std::unique_ptr<Format> spFormat;
264 | 
265 |     switch (options.id)
266 |     {
267 |     case FormatId::LZ:
268 |         spFormat = std::make_unique<FormatLZ>(options);
269 |         break;
270 | 
271 |     case FormatId::E1:
272 |         spFormat = std::make_unique<FormatE1>(options);
273 |         break;
274 | 
275 |     case FormatId::E1ZX:
276 |         spFormat = std::make_unique<FormatE1ZX>(options);
277 |         break;
278 | 
279 |     case FormatId::BX2:
280 |         spFormat = std::make_unique<FormatBX2>(options);
281 |         break;
282 | 
283 |     case FormatId::UE2:
284 |         spFormat = std::make_unique<FormatUE2>(options);
285 |         break;
286 |     }
287 | 
288 |     // Read input file.
289 | 
290 |     std::vector<uint8_t> inputStream = ReadFile(inputName.c_str());
291 |     if (inputStream.size() == 0)
292 |     {
293 |         return 1;
294 |     }
295 | 
296 |     // Compress the input stream.
297 | 
298 |     BitStream packedStream = Compress(inputStream.data(), static_cast<uint16_t>(inputStream.size()), *spFormat.get());
299 |     if (packedStream.Size() == 0)
300 |     {
301 |         PrintError(ErrorId::CompressionFailed);
302 |         return 1;
303 |     }
304 | 
305 |     if (packedStream.Size() >= inputStream.size())
306 |     {
307 |         PrintWarning(WarningId::NoSizeGain);
308 |     }
309 | 
310 |     if (options.id == FormatId::E1ZX && packedStream.IssueCarryWarning())
311 |     {
312 |         PrintWarning(WarningId::ZxExplicitCarry);
313 |     }
314 | 
315 |     // Write output file.
316 | 
317 |     if (!WriteFile(outputName.c_str(), packedStream.Data(), packedStream.Size()))
318 |     {
319 |         return 1;    
320 |     }
321 | 
322 |     return 0;
323 | }
324 | 


--------------------------------------------------------------------------------
/src/OptimalParser.cpp:
--------------------------------------------------------------------------------
 1 | // Copyright (c) 2021, Milos "baze" Bazelides
 2 | // This code is licensed under the BSD 2-Clause License.
 3 | 
 4 | #include "OptimalParser.h"
 5 | #include "PrefixMatcher.h"
 6 | 
 7 | struct PathNode
 8 | {
 9 |     uint32_t cost = 0xFFFFFFFF;
10 |     uint16_t length = 0;
11 |     uint16_t offset = 0;
12 | };
13 | 
14 | std::vector<ParseStep> Parse(const uint8_t* pInput, uint16_t inputSize, const Format& format)
15 | {
16 |     PrefixMatcher matcher(
17 |         pInput,
18 |         inputSize,
19 |         format.MinMatchLength(),
20 |         format.MaxMatchLength(),
21 |         format.MaxMatchOffset()
22 |     );
23 | 
24 |     std::vector<PathNode> nodes(inputSize + 1);
25 |     nodes[0].cost = 0;
26 | 
27 |     for (uint16_t inputPos = 0; inputPos < inputSize; inputPos++)
28 |     {
29 |         // Consider all available literals.
30 | 
31 |         uint16_t maxLength = std::min<uint16_t>(inputSize - inputPos, format.MaxLiteralLength());
32 | 
33 |         for (uint16_t length = 1; length <= maxLength; length++)
34 |         {
35 |             uint32_t cost = nodes[inputPos].cost + format.GetLiteralCost(length);
36 | 
37 |             if (cost < nodes[inputPos + length].cost)
38 |             {
39 |                 nodes[inputPos + length].cost = cost;
40 |                 nodes[inputPos + length].length = length;
41 |                 nodes[inputPos + length].offset = 0;
42 |             }
43 |         }
44 | 
45 |         // Consider all available literals.
46 | 
47 |         if (inputPos + format.MinMatchLength() > inputSize)
48 |         {
49 |             continue;
50 |         }
51 | 
52 |         for (const Match& match: matcher.FindMatches(inputPos))
53 |         {
54 |             uint32_t cost = nodes[inputPos].cost + format.GetMatchCost(match.length, match.offset);
55 | 
56 |             if (cost < nodes[inputPos + match.length].cost)
57 |             {
58 |                 nodes[inputPos + match.length].cost = cost;
59 |                 nodes[inputPos + match.length].length = match.length;
60 |                 nodes[inputPos + match.length].offset = match.offset;
61 |             }
62 |         }
63 |     }
64 | 
65 |     // Backtrack and reconstruct the optimal parse sequence.
66 | 
67 |     std::vector<ParseStep> parse;
68 | 
69 |     while (inputSize)
70 |     {
71 |         parse.emplace_back(nodes[inputSize].length, nodes[inputSize].offset);
72 |         inputSize -= nodes[inputSize].length;
73 |     }
74 | 
75 |     std::reverse(parse.begin(), parse.end());
76 | 
77 |     return parse;
78 | }
79 | 


--------------------------------------------------------------------------------
/src/OptimalParser.h:
--------------------------------------------------------------------------------
 1 | // Copyright (c) 2021, Milos "baze" Bazelides
 2 | // This code is licensed under the BSD 2-Clause License.
 3 | 
 4 | #ifndef OPTIMAL_PARSER_H
 5 | #define OPTIMAL_PARSER_H
 6 | 
 7 | #include "Formats.h"
 8 | #include "CommonTypes.h"
 9 | 
10 | std::vector<ParseStep> Parse(const uint8_t* pInput, uint16_t inputSize, const Format& format);
11 | 
12 | #endif // OPTIMAL_PARSER_H
13 | 


--------------------------------------------------------------------------------
/src/PrefixMatcher.cpp:
--------------------------------------------------------------------------------
  1 | // Copyright (c) 2025, Milos "baze" Bazelides
  2 | // This code is licensed under the BSD 2-Clause License.
  3 | 
  4 | #include <cassert>
  5 | #include <algorithm>
  6 | #include "PrefixMatcher.h"
  7 | 
  8 | PrefixMatcher::PrefixMatcher(const uint8_t* pInput, uint16_t inputSize, uint16_t minMatchLength, uint16_t maxMatchLength, uint16_t maxMatchOffset):
  9 |     mInputPtr{pInput},
 10 |     mInputSize{inputSize},
 11 |     mMinMatchLength{minMatchLength},
 12 |     mMaxMatchLength{maxMatchLength},
 13 |     mMaxMatchOffset{maxMatchOffset}
 14 | {
 15 |     // Initialize lists that track past positions of single bytes and 2-byte sequences in the input.
 16 | 
 17 |     std::vector<std::vector<uint16_t>> byteOccurrences(256);
 18 |     mBytePositions = std::make_unique<std::vector<uint16_t>[]>(inputSize);
 19 | 
 20 |     for (uint16_t inputPos = 0; inputPos < inputSize; inputPos++)
 21 |     {
 22 |         uint8_t byte = pInput[inputPos];
 23 |         uint16_t windowPos = inputPos - std::min<uint16_t>(inputPos, maxMatchOffset);
 24 | 
 25 |         for (auto i = byteOccurrences[byte].rbegin(); i != byteOccurrences[byte].rend(); i++)
 26 |         {
 27 |             if (*i < windowPos)
 28 |                 break;
 29 | 
 30 |             mBytePositions[inputPos].emplace_back(*i);
 31 |         }
 32 | 
 33 |         byteOccurrences[byte].emplace_back(inputPos);
 34 |     }
 35 | 
 36 |     std::vector<std::vector<uint16_t>> matchOccurrences(65536);
 37 |     mLongestMatches = std::make_unique<std::vector<Match>[]>(inputSize);
 38 | 
 39 |     for (uint16_t inputPos = 0; inputPos < inputSize - 1; inputPos++)
 40 |     {
 41 |         uint16_t match = (pInput[inputPos + 1] << 8) | pInput[inputPos];
 42 |         uint16_t windowPos = inputPos - std::min<uint16_t>(inputPos, maxMatchOffset);
 43 | 
 44 |         for (auto i = matchOccurrences[match].rbegin(); i != matchOccurrences[match].rend(); i++)
 45 |         {
 46 |             if (*i < windowPos)
 47 |                 break;
 48 | 
 49 |             mLongestMatches[inputPos].emplace_back(0, *i);
 50 |         }
 51 | 
 52 |         matchOccurrences[match].emplace_back(inputPos);
 53 |     }
 54 | 
 55 |     // Calculate the longest available match length at each position.
 56 | 
 57 |     for (uint16_t inputPos = 0; inputPos < inputSize; inputPos++)
 58 |     {
 59 |         for (Match& match: mLongestMatches[inputPos])
 60 |         {
 61 |             match.length = GetMatchLength(inputPos, match.offset);
 62 |         }
 63 |     }
 64 | }
 65 | 
 66 | std::vector<Match> PrefixMatcher::FindMatches(uint16_t inputPos, bool allowBytes) const
 67 | {
 68 |     std::vector<Match> matches;
 69 | 
 70 |     // Single-byte matches help set up useful repeat offsets.
 71 | 
 72 |     if (allowBytes)
 73 |     {
 74 |         for (uint16_t bytePos: mBytePositions[inputPos])
 75 |         {
 76 |             matches.emplace_back(1, inputPos - bytePos);
 77 |         }
 78 |     }
 79 | 
 80 |     // In this case, the precomputed match.offset is the absolute input position.
 81 | 
 82 |     for (const Match& match: mLongestMatches[inputPos])
 83 |     {
 84 |         uint16_t offset = inputPos - match.offset;
 85 | 
 86 |         for (uint16_t length = mMinMatchLength; length <= match.length; length++)
 87 |         {
 88 |             matches.emplace_back(length, offset);
 89 |         }
 90 |     }
 91 | 
 92 |     return matches;
 93 | }
 94 | 
 95 | Match PrefixMatcher::FindLongestMatch(uint16_t inputPos) const
 96 | {
 97 |     uint16_t maxLength = mMinMatchLength - 1;
 98 |     uint16_t maxOffset = 0;
 99 | 
100 |     for (const Match& match: mLongestMatches[inputPos])
101 |     {
102 |         if (match.length > maxLength)
103 |         {
104 |             maxLength = match.length;
105 |             maxOffset = inputPos - match.offset;
106 |         }
107 |     }
108 | 
109 |     return {maxLength, maxOffset};
110 | }
111 | 
112 | uint16_t PrefixMatcher::GetMatchLength(uint16_t inputPos, uint16_t matchPos) const
113 | {
114 |     const uint8_t* pInput = mInputPtr + inputPos;
115 |     const uint8_t* pMatch = mInputPtr + matchPos;
116 |     const uint8_t* pEnd = pInput + std::min<uint16_t>(mInputSize - inputPos, mMaxMatchLength);
117 | 
118 |     return static_cast<uint16_t>(std::mismatch(pInput, pEnd, pMatch).first - pInput);
119 | }
120 | 


--------------------------------------------------------------------------------
/src/PrefixMatcher.h:
--------------------------------------------------------------------------------
 1 | // Copyright (c) 2025, Milos "baze" Bazelides
 2 | // This code is licensed under the BSD 2-Clause License.
 3 | 
 4 | #ifndef PREFIX_MATCHER_H
 5 | #define PREFIX_MATCHER_H
 6 | 
 7 | #include <memory>
 8 | #include <vector>
 9 | #include "CommonTypes.h"
10 | 
11 | class PrefixMatcher
12 | {
13 | public:
14 | 
15 |     PrefixMatcher() = delete;
16 | 
17 |     PrefixMatcher(
18 |         const uint8_t* pInput,
19 |         uint16_t inputSize,
20 |         uint16_t minMatchLength,
21 |         uint16_t maxMatchLength,
22 |         uint16_t maxMatchOffset
23 |     );
24 | 
25 |     std::vector<Match> FindMatches(uint16_t inputPos, bool allowBytes = false) const;
26 |     Match FindLongestMatch(uint16_t inputPos) const;
27 | 
28 | private:
29 | 
30 |     uint16_t GetMatchLength(uint16_t inputPos, uint16_t matchPos) const;
31 | 
32 |     const uint8_t* mInputPtr;
33 |     const uint16_t mInputSize;
34 | 
35 |     const uint16_t mMinMatchLength;
36 |     const uint16_t mMaxMatchLength;
37 |     const uint16_t mMaxMatchOffset;
38 | 
39 |     std::unique_ptr<std::vector<uint16_t>[]> mBytePositions;
40 |     std::unique_ptr<std::vector<Match>[]> mLongestMatches;
41 | };
42 | 
43 | #endif // PREFIX_MATCHER_H
44 | 


--------------------------------------------------------------------------------
/src/UniversalCodes.cpp:
--------------------------------------------------------------------------------
  1 | // Copyright (c) 2021, Milos "baze" Bazelides
  2 | // This code is licensed under the BSD 2-Clause License.
  3 | 
  4 | #include <cassert>
  5 | #include "UniversalCodes.h"
  6 | 
  7 | // Elias-Gamma 1..N encoding (interleaved format).
  8 | 
  9 | // 1: 0
 10 | // 2: 100
 11 | // 3: 110
 12 | // 4: 10100
 13 | // 5: 10110
 14 | // 6: 11100
 15 | // 7: 11110
 16 | // 8: 1010100
 17 | 
 18 | uint32_t GetElias1Cost(uint32_t value)
 19 | {
 20 |     assert(value > 0);
 21 | 
 22 |     uint32_t cost = 1;
 23 |     while (value >>= 1)
 24 |     {
 25 |         cost += 2;
 26 |     }
 27 | 
 28 |     return cost;
 29 | }
 30 | 
 31 | void EncodeElias1(BitStream& stream, uint32_t value)
 32 | {
 33 |     assert(value > 0);
 34 | 
 35 |     uint32_t copy = value;
 36 |     uint32_t mask = 1;
 37 | 
 38 |     while (copy >>= 1)
 39 |     {
 40 |         mask <<= 1;
 41 |     }
 42 | 
 43 |     while (mask >>= 1)
 44 |     {
 45 |         stream.WriteBit(1);
 46 |         stream.WriteBit(value & mask);
 47 |     }
 48 | 
 49 |     stream.WriteBit(0);
 50 | }
 51 | 
 52 | uint32_t DecodeElias1(BitStream& stream)
 53 | {
 54 |     uint32_t value = 1;
 55 | 
 56 |     while (stream.ReadBit())
 57 |     {
 58 |         value = (value << 1) | stream.ReadBit();
 59 |     }
 60 | 
 61 |     return value;
 62 | }
 63 | 
 64 | // Elias-Gamma 2..N encoding (interleaved format).
 65 | 
 66 | // 2: 00
 67 | // 3: 10
 68 | // 4: 0100
 69 | // 5: 0110
 70 | // 6: 1100
 71 | // 7: 1110
 72 | // 8: 010100
 73 | // 9: 010110
 74 | 
 75 | uint32_t GetElias2Cost(uint32_t value)
 76 | {
 77 |     assert(value > 1);
 78 | 
 79 |     uint32_t mask = ~3;
 80 |     uint32_t count = 1;
 81 | 
 82 |     while (value & mask)
 83 |     {
 84 |         mask <<= 1;
 85 |         count++;
 86 |     }
 87 | 
 88 |     return count << 1;
 89 | }
 90 | 
 91 | void EncodeElias2(BitStream& stream, uint32_t value)
 92 | {
 93 |     assert(value > 1);
 94 | 
 95 |     uint32_t copy = value >> 1;
 96 |     uint32_t mask = 1;
 97 | 
 98 |     while (copy >>= 1)
 99 |     {
100 |         mask <<= 1;
101 |     }
102 | 
103 |     while (mask)
104 |     {
105 |         stream.WriteBit(value & mask);
106 |         mask >>= 1;
107 |         if (mask)
108 |         {
109 |             stream.WriteBit(1);
110 |         }
111 |     }
112 | 
113 |     stream.WriteBit(0);
114 | }
115 | 
116 | uint32_t DecodeElias2(BitStream& stream)
117 | {
118 |     uint32_t value = 1;
119 | 
120 |     do
121 |     {
122 |         value = (value << 1) | stream.ReadBit();
123 |     }
124 |     while (stream.ReadBit());
125 | 
126 |     return value;
127 | }
128 | 
129 | // Unary encoding.
130 | 
131 | // 0: 0
132 | // 1: 10
133 | // 2: 110
134 | // 3: 1110
135 | // 4: 11110
136 | // 5: 111110
137 | // 6: 1111110
138 | // 7: 11111110
139 | 
140 | uint32_t GetUnaryCost(uint32_t value)
141 | {
142 |     return value + 1;
143 | }
144 | 
145 | void EncodeUnary(BitStream& stream, uint32_t value)
146 | {
147 |     for (uint32_t i = 0; i < value; i++)
148 |     {
149 |         stream.WriteBit(1);
150 |     }
151 | 
152 |     stream.WriteBit(0);
153 | }
154 | 
155 | uint32_t DecodeUnary(BitStream& stream)
156 | {
157 |     uint32_t value = 0;
158 | 
159 |     while (stream.ReadBit())
160 |     {
161 |         value++;
162 |     }
163 | 
164 |     return value;
165 | }
166 | 
167 | // Rice encoding (K = 1).
168 | 
169 | // 0: 00
170 | // 1: 01
171 | // 2: 100
172 | // 3: 101
173 | // 4: 1100
174 | // 5: 1101
175 | // 6: 11100
176 | // 7: 11101
177 | 
178 | uint32_t GetRiceCost(uint32_t value)
179 | {
180 |     return (value >> 1) + 2;
181 | }
182 | 
183 | void EncodeRice(BitStream& stream, uint32_t value)
184 | {
185 |     for (uint32_t i = 0; i < value >> 1; i++)
186 |     {
187 |         stream.WriteBit(1);
188 |     }
189 | 
190 |     stream.WriteBit(0);
191 |     stream.WriteBit(value & 1);
192 | }
193 | 
194 | uint32_t DecodeRice(BitStream& stream)
195 | {
196 |     uint32_t value = 0;
197 | 
198 |     while (stream.ReadBit())
199 |     {
200 |         value++;
201 |     }
202 | 
203 |     return (value << 1) | stream.ReadBit();
204 | }
205 | 
206 | // Vbinary 2x2 encoding.
207 | 
208 | // 0: 00
209 | // 1: 01
210 | // 2: 10
211 | // 3: 1100
212 | // 4: 1101
213 | // 5: 1110
214 | // 6: 111100
215 | // 7: 111101
216 | 
217 | uint32_t GetVbinCost(uint32_t value)
218 | {
219 |     return (value / 3 + 1) << 1;
220 | }
221 | 
222 | void EncodeVbin(BitStream& stream, uint32_t value)
223 | {
224 |     uint32_t count = value / 3;
225 | 
226 |     for (uint32_t i = 0; i < count; i++)
227 |     {
228 |         stream.WriteBit(1);
229 |         stream.WriteBit(1);
230 |         value -= 3;
231 |     }
232 | 
233 |     stream.WriteBit(value & 2);
234 |     stream.WriteBit(value & 1);
235 | }
236 | 
237 | uint32_t DecodeVbin(BitStream& stream)
238 | {
239 |     uint32_t value = 0;
240 | 
241 |     while (true)
242 |     {
243 |         uint32_t bits = stream.ReadBit();
244 |         bits = (bits << 1) | stream.ReadBit();
245 |         value += bits;
246 | 
247 |         if ((bits & 3) < 3)
248 |         {
249 |             break;
250 |         }
251 |     }
252 | 
253 |     return value;
254 | }
255 | 
256 | // Plain binary encoding (the number of bits is explicit).
257 | 
258 | void EncodeRaw(BitStream& stream, uint32_t value, uint32_t numBits)
259 | {
260 |     assert(numBits > 0);
261 | 
262 |     uint32_t mask = 1 << (numBits - 1);
263 | 
264 |     while (mask)
265 |     {
266 |         stream.WriteBit(value & mask);
267 |         mask >>= 1;
268 |     }
269 | }
270 | 
271 | uint32_t DecodeRaw(BitStream& stream, uint32_t numBits)
272 | {
273 |     assert(numBits > 0);
274 | 
275 |     uint32_t value = 0;
276 | 
277 |     while (numBits--)
278 |     {
279 |         value = (value << 1) | stream.ReadBit();
280 |     }
281 | 
282 |     return value;
283 | }
284 | 
285 | // These methods are only required by the E1ZX format.
286 | 
287 | void EncodeElias1Neg(BitStream& stream, uint32_t value)
288 | {
289 |     assert(value > 0);
290 | 
291 |     uint32_t mask = ~1;
292 |     uint32_t count = 0;
293 | 
294 |     while (value & mask)
295 |     {
296 |         mask <<= 1;
297 |         count++;
298 |     }
299 | 
300 |     mask = 1 << count;
301 | 
302 |     while (mask >>= 1)
303 |     {
304 |         stream.WriteBitNeg(1);
305 |         stream.WriteBitNeg(value & mask);
306 |     }
307 | 
308 |     stream.WriteBitNeg(0);
309 | }
310 | 
311 | uint32_t DecodeElias1Neg(BitStream& stream)
312 | {
313 |     uint32_t value = 1;
314 | 
315 |     while (stream.ReadBitNeg())
316 |     {
317 |         value = (value << 1) | stream.ReadBitNeg();
318 |     }
319 | 
320 |     return value;
321 | }
322 | 


--------------------------------------------------------------------------------
/src/UniversalCodes.h:
--------------------------------------------------------------------------------
 1 | // Copyright (c) 2021, Milos "baze" Bazelides
 2 | // This code is licensed under the BSD 2-Clause License.
 3 | 
 4 | #ifndef UNIVERSAL_CODES_H
 5 | #define UNIVERSAL_CODES_H
 6 | 
 7 | #include "BitStream.h"
 8 | 
 9 | uint32_t GetElias1Cost(uint32_t value);
10 | void EncodeElias1(BitStream& stream, uint32_t value);
11 | uint32_t DecodeElias1(BitStream& stream);
12 | 
13 | uint32_t GetElias2Cost(uint32_t value);
14 | void EncodeElias2(BitStream& stream, uint32_t value);
15 | uint32_t DecodeElias2(BitStream& stream);
16 | 
17 | uint32_t GetUnaryCost(uint32_t value);
18 | void EncodeUnary(BitStream& stream, uint32_t value);
19 | uint32_t DecodeUnary(BitStream& stream);
20 | 
21 | uint32_t GetRiceCost(uint32_t value);
22 | void EncodeRice(BitStream& stream, uint32_t value);
23 | uint32_t DecodeRice(BitStream& stream);
24 | 
25 | uint32_t GetVbinCost(uint32_t value);
26 | void EncodeVbin(BitStream& stream, uint32_t value);
27 | uint32_t DecodeVbin(BitStream& stream);
28 | 
29 | void EncodeRaw(BitStream& stream, uint32_t value, uint32_t bits);
30 | uint32_t DecodeRaw(BitStream& stream, uint32_t bits);
31 | 
32 | // These methods are only required by the E1ZX format.
33 | 
34 | void EncodeElias1Neg(BitStream& stream, uint32_t value);
35 | uint32_t DecodeElias1Neg(BitStream& stream);
36 | 
37 | #endif // UNIVERSAL_CODES_H
38 | 


--------------------------------------------------------------------------------
/vcxproj/bzpack.sln:
--------------------------------------------------------------------------------
 1 | 
 2 | Microsoft Visual Studio Solution File, Format Version 12.00
 3 | # Visual Studio Version 16
 4 | VisualStudioVersion = 16.0.31424.327
 5 | MinimumVisualStudioVersion = 10.0.40219.1
 6 | Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "bzpack", "bzpack.vcxproj", "{FDDF984D-70FB-4BB9-BC6C-BC658997C98B}"
 7 | EndProject
 8 | Global
 9 | 	GlobalSection(SolutionConfigurationPlatforms) = preSolution
10 | 		Debug|x64 = Debug|x64
11 | 		Release|x64 = Release|x64
12 | 	EndGlobalSection
13 | 	GlobalSection(ProjectConfigurationPlatforms) = postSolution
14 | 		{FDDF984D-70FB-4BB9-BC6C-BC658997C98B}.Debug|x64.ActiveCfg = Debug|x64
15 | 		{FDDF984D-70FB-4BB9-BC6C-BC658997C98B}.Debug|x64.Build.0 = Debug|x64
16 | 		{FDDF984D-70FB-4BB9-BC6C-BC658997C98B}.Release|x64.ActiveCfg = Release|x64
17 | 		{FDDF984D-70FB-4BB9-BC6C-BC658997C98B}.Release|x64.Build.0 = Release|x64
18 | 	EndGlobalSection
19 | 	GlobalSection(SolutionProperties) = preSolution
20 | 		HideSolutionNode = FALSE
21 | 	EndGlobalSection
22 | 	GlobalSection(ExtensibilityGlobals) = postSolution
23 | 		SolutionGuid = {59046676-2D8D-4DAD-A385-F3AB32A93785}
24 | 	EndGlobalSection
25 | EndGlobal
26 | 


--------------------------------------------------------------------------------
/vcxproj/bzpack.vcxproj:
--------------------------------------------------------------------------------
  1 | <?xml version="1.0" encoding="utf-8"?>
  2 | <Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  3 |   <ItemGroup Label="ProjectConfigurations">
  4 |     <ProjectConfiguration Include="Debug|Win32">
  5 |       <Configuration>Debug</Configuration>
  6 |       <Platform>Win32</Platform>
  7 |     </ProjectConfiguration>
  8 |     <ProjectConfiguration Include="Release|Win32">
  9 |       <Configuration>Release</Configuration>
 10 |       <Platform>Win32</Platform>
 11 |     </ProjectConfiguration>
 12 |     <ProjectConfiguration Include="Debug|x64">
 13 |       <Configuration>Debug</Configuration>
 14 |       <Platform>x64</Platform>
 15 |     </ProjectConfiguration>
 16 |     <ProjectConfiguration Include="Release|x64">
 17 |       <Configuration>Release</Configuration>
 18 |       <Platform>x64</Platform>
 19 |     </ProjectConfiguration>
 20 |   </ItemGroup>
 21 |   <ItemGroup>
 22 |     <ClCompile Include="..\src\BitStream.cpp" />
 23 |     <ClCompile Include="..\src\Compressor.cpp" />
 24 |     <ClCompile Include="..\src\Decompressor.cpp" />
 25 |     <ClCompile Include="..\src\DijkstraParser.cpp" />
 26 |     <ClCompile Include="..\src\Main.cpp" />
 27 |     <ClCompile Include="..\src\PrefixMatcher.cpp" />
 28 |     <ClCompile Include="..\src\OptimalParser.cpp" />
 29 |     <ClCompile Include="..\src\UniversalCodes.cpp" />
 30 |   </ItemGroup>
 31 |   <ItemGroup>
 32 |     <ClInclude Include="..\src\BitStream.h" />
 33 |     <ClInclude Include="..\src\Compressor.h" />
 34 |     <ClInclude Include="..\src\DijkstraParser.h" />
 35 |     <ClInclude Include="..\src\Formats.h" />
 36 |     <ClInclude Include="..\src\PrefixMatcher.h" />
 37 |     <ClInclude Include="..\src\OptimalParser.h" />
 38 |     <ClInclude Include="..\src\CommonTypes.h" />
 39 |     <ClInclude Include="..\src\UniversalCodes.h" />
 40 |   </ItemGroup>
 41 |   <PropertyGroup Label="Globals">
 42 |     <VCProjectVersion>16.0</VCProjectVersion>
 43 |     <Keyword>Win32Proj</Keyword>
 44 |     <ProjectGuid>{fddf984d-70fb-4bb9-bc6c-bc658997c98b}</ProjectGuid>
 45 |     <RootNamespace>bzpack</RootNamespace>
 46 |     <WindowsTargetPlatformVersion>10.0</WindowsTargetPlatformVersion>
 47 |   </PropertyGroup>
 48 |   <Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
 49 |   <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
 50 |     <ConfigurationType>Application</ConfigurationType>
 51 |     <UseDebugLibraries>true</UseDebugLibraries>
 52 |     <PlatformToolset>v143</PlatformToolset>
 53 |     <CharacterSet>Unicode</CharacterSet>
 54 |   </PropertyGroup>
 55 |   <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
 56 |     <ConfigurationType>Application</ConfigurationType>
 57 |     <UseDebugLibraries>false</UseDebugLibraries>
 58 |     <PlatformToolset>v143</PlatformToolset>
 59 |     <WholeProgramOptimization>true</WholeProgramOptimization>
 60 |     <CharacterSet>Unicode</CharacterSet>
 61 |   </PropertyGroup>
 62 |   <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="Configuration">
 63 |     <ConfigurationType>Application</ConfigurationType>
 64 |     <UseDebugLibraries>true</UseDebugLibraries>
 65 |     <PlatformToolset>v143</PlatformToolset>
 66 |     <CharacterSet>Unicode</CharacterSet>
 67 |   </PropertyGroup>
 68 |   <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="Configuration">
 69 |     <ConfigurationType>Application</ConfigurationType>
 70 |     <UseDebugLibraries>false</UseDebugLibraries>
 71 |     <PlatformToolset>v143</PlatformToolset>
 72 |     <WholeProgramOptimization>true</WholeProgramOptimization>
 73 |     <CharacterSet>Unicode</CharacterSet>
 74 |   </PropertyGroup>
 75 |   <Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
 76 |   <ImportGroup Label="ExtensionSettings">
 77 |   </ImportGroup>
 78 |   <ImportGroup Label="Shared">
 79 |   </ImportGroup>
 80 |   <ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
 81 |     <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
 82 |   </ImportGroup>
 83 |   <ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
 84 |     <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
 85 |   </ImportGroup>
 86 |   <ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
 87 |     <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
 88 |   </ImportGroup>
 89 |   <ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
 90 |     <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
 91 |   </ImportGroup>
 92 |   <PropertyGroup Label="UserMacros" />
 93 |   <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
 94 |     <LinkIncremental>true</LinkIncremental>
 95 |   </PropertyGroup>
 96 |   <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
 97 |     <LinkIncremental>false</LinkIncremental>
 98 |   </PropertyGroup>
 99 |   <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
100 |     <LinkIncremental>true</LinkIncremental>
101 |     <OutDir>..\build\$(Platform)\$(Configuration)\</OutDir>
102 |     <IntDir>..\build\$(Platform)\$(Configuration)\</IntDir>
103 |   </PropertyGroup>
104 |   <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
105 |     <LinkIncremental>false</LinkIncremental>
106 |     <OutDir>..\build\$(Platform)\$(Configuration)\</OutDir>
107 |     <IntDir>..\build\$(Platform)\$(Configuration)\</IntDir>
108 |   </PropertyGroup>
109 |   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
110 |     <ClCompile>
111 |       <WarningLevel>Level3</WarningLevel>
112 |       <SDLCheck>true</SDLCheck>
113 |       <PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
114 |       <ConformanceMode>true</ConformanceMode>
115 |       <LanguageStandard>Default</LanguageStandard>
116 |     </ClCompile>
117 |     <Link>
118 |       <SubSystem>Console</SubSystem>
119 |       <GenerateDebugInformation>true</GenerateDebugInformation>
120 |     </Link>
121 |   </ItemDefinitionGroup>
122 |   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
123 |     <ClCompile>
124 |       <WarningLevel>Level3</WarningLevel>
125 |       <FunctionLevelLinking>true</FunctionLevelLinking>
126 |       <IntrinsicFunctions>true</IntrinsicFunctions>
127 |       <SDLCheck>true</SDLCheck>
128 |       <PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
129 |       <ConformanceMode>true</ConformanceMode>
130 |       <LanguageStandard>Default</LanguageStandard>
131 |     </ClCompile>
132 |     <Link>
133 |       <SubSystem>Console</SubSystem>
134 |       <EnableCOMDATFolding>true</EnableCOMDATFolding>
135 |       <OptimizeReferences>true</OptimizeReferences>
136 |       <GenerateDebugInformation>true</GenerateDebugInformation>
137 |     </Link>
138 |   </ItemDefinitionGroup>
139 |   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
140 |     <ClCompile>
141 |       <WarningLevel>Level3</WarningLevel>
142 |       <SDLCheck>true</SDLCheck>
143 |       <PreprocessorDefinitions>_DEBUG;_CONSOLE;_CRT_SECURE_NO_WARNINGS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
144 |       <ConformanceMode>true</ConformanceMode>
145 |       <LanguageStandard>Default</LanguageStandard>
146 |     </ClCompile>
147 |     <Link>
148 |       <SubSystem>Console</SubSystem>
149 |       <GenerateDebugInformation>true</GenerateDebugInformation>
150 |     </Link>
151 |   </ItemDefinitionGroup>
152 |   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
153 |     <ClCompile>
154 |       <WarningLevel>Level3</WarningLevel>
155 |       <FunctionLevelLinking>true</FunctionLevelLinking>
156 |       <IntrinsicFunctions>true</IntrinsicFunctions>
157 |       <SDLCheck>true</SDLCheck>
158 |       <PreprocessorDefinitions>NDEBUG;_CONSOLE;_CRT_SECURE_NO_WARNINGS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
159 |       <ConformanceMode>true</ConformanceMode>
160 |       <LanguageStandard>Default</LanguageStandard>
161 |     </ClCompile>
162 |     <Link>
163 |       <SubSystem>Console</SubSystem>
164 |       <EnableCOMDATFolding>true</EnableCOMDATFolding>
165 |       <OptimizeReferences>true</OptimizeReferences>
166 |       <GenerateDebugInformation>true</GenerateDebugInformation>
167 |     </Link>
168 |   </ItemDefinitionGroup>
169 |   <Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
170 |   <ImportGroup Label="ExtensionTargets">
171 |   </ImportGroup>
172 | </Project>


--------------------------------------------------------------------------------
/vcxproj/bzpack.vcxproj.filters:
--------------------------------------------------------------------------------
 1 | <?xml version="1.0" encoding="utf-8"?>
 2 | <Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
 3 |   <ItemGroup>
 4 |     <ClCompile Include="..\src\BitStream.cpp" />
 5 |     <ClCompile Include="..\src\Compressor.cpp" />
 6 |     <ClCompile Include="..\src\Main.cpp" />
 7 |     <ClCompile Include="..\src\OptimalParser.cpp" />
 8 |     <ClCompile Include="..\src\UniversalCodes.cpp" />
 9 |     <ClCompile Include="..\src\Decompressor.cpp" />
10 |     <ClCompile Include="..\src\PrefixMatcher.cpp" />
11 |     <ClCompile Include="..\src\DijkstraParser.cpp" />
12 |   </ItemGroup>
13 |   <ItemGroup>
14 |     <ClInclude Include="..\src\BitStream.h" />
15 |     <ClInclude Include="..\src\Compressor.h" />
16 |     <ClInclude Include="..\src\OptimalParser.h" />
17 |     <ClInclude Include="..\src\UniversalCodes.h" />
18 |     <ClInclude Include="..\src\Formats.h" />
19 |     <ClInclude Include="..\src\PrefixMatcher.h" />
20 |     <ClInclude Include="..\src\CommonTypes.h" />
21 |     <ClInclude Include="..\src\DijkstraParser.h" />
22 |   </ItemGroup>
23 | </Project>


--------------------------------------------------------------------------------