├── .dockerignore ├── .gitignore ├── .mbedignore ├── Dockerfile.valgrind ├── LICENSE ├── Makefile ├── README.md ├── cli └── janpatch-cli.c ├── demo ├── blinky-k64f-new.bin ├── blinky-k64f-old.bin ├── blinky-k64f-reverse.patch └── blinky-k64f.patch ├── integration-tests ├── run-tests.js └── source │ ├── 00.bin │ ├── FF.bin │ ├── blinky-f411re.bin │ ├── blinky-ff1705.bin │ ├── blinky-k64f.bin │ ├── pg16328-images.epub │ ├── pg1952-images.epub │ ├── pg2591-images.epub │ ├── pg844-images.epub │ └── pg98-images.epub ├── janpatch.h └── jdiff-binary-sources ├── .gitignore ├── build-all.sh ├── jdiff07.tgz ├── jdiff081.tgz ├── jdiff085.tgz └── source.txt /.dockerignore: -------------------------------------------------------------------------------- 1 | jdiff-binary-sources/jdiff07/ 2 | jdiff-binary-sources/jdiff081/ 3 | jdiff-binary-sources/jdiff085/ 4 | integration-tests/target/ 5 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | janpatch-cli 2 | janpatch-cli.dSYM/ 3 | blinky-k64f-patched.bin 4 | integration-tests/target/ 5 | .DS_Store 6 | -------------------------------------------------------------------------------- /.mbedignore: -------------------------------------------------------------------------------- 1 | cli/ 2 | integration-tests/ 3 | demo/ 4 | -------------------------------------------------------------------------------- /Dockerfile.valgrind: -------------------------------------------------------------------------------- 1 | # syntax = docker/dockerfile:experimental@sha256:3c244c0c6fc9d6aa3ddb73af4264b3a23597523ac553294218c13735a2c6cf79 2 | # 20.04 3 | FROM ubuntu:focal-20241011 4 | 5 | ARG SKAFFOLD_MODE 6 | ARG DEBIAN_FRONTEND=noninteractive 7 | 8 | WORKDIR /app 9 | 10 | RUN apt update && apt install -y build-essential valgrind 11 | 12 | # Rest of the files 13 | COPY . ./ 14 | RUN make 15 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2017 ARM Ltd. 2 | 3 | Licensed under the Apache License, Version 2.0 (the "License"); 4 | you may not use this file except in compliance with the License. 5 | You may obtain a copy of the License at 6 | 7 | http://www.apache.org/licenses/LICENSE-2.0 8 | 9 | Unless required by applicable law or agreed to in writing, software 10 | distributed under the License is distributed on an "AS IS" BASIS, 11 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | See the License for the specific language governing permissions and 13 | limitations under the License. 14 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | NAME = janpatch-cli 2 | 3 | CC ?= gcc 4 | CFLAGS ?= -Wall 5 | 6 | MACROS += -DJANPATCH_STREAM=FILE 7 | CFLAGS += -I. 8 | 9 | all: build 10 | 11 | .PHONY: build clean 12 | 13 | build: 14 | $(CC) $(MACROS) $(CFLAGS) cli/janpatch-cli.c -o $(NAME) 15 | 16 | clean: 17 | rm $(NAME) -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Jojo AlterNative Patch (JANPatch) 2 | 3 | Memory-efficient library which can apply [JojoDiff](http://jojodiff.sourceforge.net) patch files. Very useful for embedded development, for example for delta firmware updates. Written in C, and can be backed by anything that offers basic I/O primitives. 4 | 5 | This is an implementation from scratch, done by reverse engineering the diff file format, and inspecting the diff generation code. It's not a derived work from jptch (the patch application in JojoDiff). The reason for this is that JojoDiff is licensed under GPLv3, which does not allow linking in the library in non-GPL applications. JANPatch is licensed under Apache 2.0. 6 | 7 | For an example of using this library on an embedded system, see [binary-diff-mbedos5](https://github.com/janjongboom/binary-diff-mbedos5). 8 | 9 | ## Usage (CLI) 10 | 11 | On POSIX systems (macOS, Linux, Cygwin under Windows): 12 | 13 | 1. Build the CLI, via: 14 | 15 | ``` 16 | $ make 17 | ``` 18 | 19 | 1. Run `./janpatch-cli` with the old file, a JojoDiff patch file, and the destination: 20 | 21 | ``` 22 | $ ./janpatch-cli demo/blinky-k64f-old.bin demo/blinky-k64f.patch ./blinky-k64f-patched.bin 23 | ``` 24 | 25 | 1. Verify that the patch was applied successfully: 26 | 27 | ``` 28 | $ diff demo/blinky-k64f-new.bin blinky-k64f-patched.bin 29 | 30 | # should return nothing 31 | ``` 32 | 33 | ## Usage (library) 34 | 35 | JANPatch is implemented in a single header file, which you can copy into your project. For portability to non-POSIX platforms you need to provide the library with function pointers to basic IO operations. These are `fread`, `fwrite`, `fseek` and (optionally) `ftell`. The file pointer for these functions is of type `JANPATCH_STREAM`, which you can set when building. 36 | 37 | The functions are defined in a context, for example on POSIX systems, you define JANPatch like this: 38 | 39 | ```cpp 40 | #define JANPATCH_STREAM FILE // use POSIX FILE 41 | 42 | #include "janpatch.h" 43 | 44 | janpatch_ctx ctx = { 45 | // fread/fwrite buffers for every file, minimum size is 1 byte 46 | // when you run on an embedded system with block size flash, set it to the size of a block for best performance 47 | { (unsigned char*)malloc(1024), 1024 }, 48 | { (unsigned char*)malloc(1024), 1024 }, 49 | { (unsigned char*)malloc(1024), 1024 }, 50 | 51 | // define functions which can perform basic IO 52 | // on POSIX, use: 53 | &fread, 54 | &fwrite, 55 | &fseek, 56 | &ftell // NOTE: passing ftell is optional, and only required when you need progress reports 57 | }; 58 | ``` 59 | 60 | Then create three objects of type `JANPATCH_STREAM` and call `janpatch` with the context: 61 | 62 | ```cpp 63 | JANPATCH_STREAM *source = fopen("source.bin", "rb"); 64 | JANPATCH_STREAM *patch = fopen("patch", "rb"); 65 | JANPATCH_STREAM *target = fopen("target.bin", "wb"); 66 | 67 | janpatch(ctx, source, patch, target); 68 | ``` 69 | 70 | For a demonstration of using this library on a non-POSIX system, see [binary-diff-mbedos5#xdot](https://github.com/janjongboom/binary-diff-mbedos5/tree/xdot). 71 | 72 | ### Reporting progress 73 | 74 | Exact progress indication is hard, as the size of the file after patching is not known, but you can get rudimentary progress (based on the pages written to the target stream) by setting the `progress` property on the context: 75 | 76 | ``` 77 | void progress(uint8_t percentage) { 78 | printf("Patch progress: %d%%\n", percentage); 79 | } 80 | 81 | ctx.progress = &progress; 82 | ``` 83 | 84 | Note that you need to have declared `ctx.ftell` for this to work. 85 | 86 | ## Generating patch files 87 | 88 | To generate patch files you'll need to build [JojoDiff](http://jojodiff.sourceforge.net) or [JDiff.js](https://github.com/janjongboom/jdiff-js). 89 | 90 | 1. Install a recent version of [Node.js](https://nodejs.org). 91 | 1. Install JDiff.js: 92 | 93 | ``` 94 | $ npm install jdiff-js -g 95 | ``` 96 | 97 | 1. Generate a patch file via: 98 | 99 | ``` 100 | $ jdiff old-file.bin new-file.bin old-to-new-file.patch 101 | ``` 102 | 103 | ## Running tests 104 | 105 | To test janpatch against a wide variety of outputs you can run the integration tests. The script will generate all possible diffs between files in the `integration-tests/source` directory, then use janpatch to patch them. 106 | 107 | To run: 108 | 109 | 1. Install a recent version of [Node.js](https://nodejs.org). 110 | 2. Build JojoDiff 0.7, 0.8.1 and 0.8.5 via: 111 | 112 | ``` 113 | sh jdiff-binary-sources/build-all.sh 114 | ``` 115 | 116 | 3. Run: 117 | 118 | ``` 119 | node integration-tests/run-tests.js --jdiff-path jdiff-binary-sources/jdiff07/src/jdiff 120 | node integration-tests/run-tests.js --jdiff-path jdiff-binary-sources/jdiff081/src/jdiff 121 | node integration-tests/run-tests.js --jdiff-path jdiff-binary-sources/jdiff085/src/jdiff 122 | ``` 123 | 124 | ## Run through Valgrind 125 | 126 | 1. Build the test container: 127 | 128 | ``` 129 | docker build -t jdiff-valgrind -f Dockerfile.valgrind . 130 | ``` 131 | 132 | 2. Run the CLI through Valgrind: 133 | 134 | ``` 135 | docker run --rm -it -v $PWD/cli/:/app/cli/ -v $PWD/integration-tests/:/app/integration-tests/ jdiff-valgrind bash -c "\ 136 | valgrind --leak-check=full ./janpatch-cli integration-tests/source/blinky-k64f.bin integration-tests/target/blinky-k64f_blinky-f411re.diff bleep.patched 137 | " 138 | ``` 139 | 140 | This should print "All heap blocks were freed -- no leaks are possible". 141 | 142 | ## Mbed OS 5 - Building without loading UART driver 143 | 144 | Error messages are printed over `printf`, which will automatically load the UART drivers on Mbed OS 5. You can mitigate this by using `debug()`, which is stripped out when building for a release profile. 145 | 146 | You can do this by declaring the `JANPATCH_ERROR` macro as such: 147 | 148 | ``` 149 | #define JANPATCH_ERROR(...) debug(__VA_ARGS__) 150 | ``` 151 | 152 | ## License 153 | 154 | Apache License version 2. See [LICENSE](LICENSE). 155 | -------------------------------------------------------------------------------- /cli/janpatch-cli.c: -------------------------------------------------------------------------------- 1 | #include 2 | #include "janpatch.h" 3 | 4 | int main(int argc, char **argv) { 5 | if (argc != 3 && argc != 4) { 6 | printf("Usage: janpatch-cli [old-file] [patch-file] ([new-file])\n"); 7 | return 1; 8 | } 9 | 10 | // Open streams 11 | FILE *old = fopen(argv[1], "rb"); 12 | FILE *patch = fopen(argv[2], "rb"); 13 | FILE *target = argc == 4 ? fopen(argv[3], "wb") : stdout; 14 | 15 | if (!old) { 16 | printf("Could not open '%s'\n", argv[1]); 17 | return 1; 18 | } 19 | if (!patch) { 20 | printf("Could not open '%s'\n", argv[2]); 21 | return 1; 22 | } 23 | if (!target) { 24 | printf("Could not open '%s'\n", argv[3]); 25 | return 1; 26 | } 27 | 28 | // janpatch_ctx contains buffers, and references to the file system functions 29 | janpatch_ctx ctx = { 30 | { (unsigned char*)malloc(1024), 1024 }, // source buffer 31 | { (unsigned char*)malloc(1024), 1024 }, // patch buffer 32 | { (unsigned char*)malloc(1024), 1024 }, // target buffer 33 | 34 | &fread, 35 | &fwrite, 36 | &fseek, 37 | &ftell 38 | }; 39 | 40 | int r = janpatch(ctx, old, patch, target); 41 | 42 | fclose(old); 43 | fclose(patch); 44 | fclose(target); 45 | 46 | free(ctx.source_buffer.buffer); 47 | free(ctx.patch_buffer.buffer); 48 | free(ctx.target_buffer.buffer); 49 | 50 | return r; 51 | } 52 | -------------------------------------------------------------------------------- /demo/blinky-k64f-new.bin: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/janjongboom/janpatch/c8ad3eeabf75640d6348a706abe7e3377a21ad87/demo/blinky-k64f-new.bin -------------------------------------------------------------------------------- /demo/blinky-k64f-old.bin: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/janjongboom/janpatch/c8ad3eeabf75640d6348a706abe7e3377a21ad87/demo/blinky-k64f-old.bin -------------------------------------------------------------------------------- /demo/blinky-k64f-reverse.patch: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/janjongboom/janpatch/c8ad3eeabf75640d6348a706abe7e3377a21ad87/demo/blinky-k64f-reverse.patch -------------------------------------------------------------------------------- /demo/blinky-k64f.patch: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/janjongboom/janpatch/c8ad3eeabf75640d6348a706abe7e3377a21ad87/demo/blinky-k64f.patch -------------------------------------------------------------------------------- /integration-tests/run-tests.js: -------------------------------------------------------------------------------- 1 | const fs = require('fs'); 2 | const Path = require('path'); 3 | const spawn = require('child_process').spawn; 4 | const crypto = require('crypto'); 5 | const util = require('util'); 6 | 7 | (async () => { 8 | try { 9 | let jdiff = 'jdiff'; 10 | if (process.argv.indexOf('--jdiff-path') > -1) { 11 | jdiff = process.argv[process.argv.indexOf('--jdiff-path') + 1]; 12 | } 13 | 14 | const files = await fs.promises.readdir(Path.join(__dirname, 'source')); 15 | if (!await pathExists(Path.join(__dirname, 'target'))) { 16 | await fs.promises.mkdir(Path.join(__dirname, 'target'), { recursive: true }); 17 | } 18 | 19 | const target_files = []; 20 | for (let s = 0; s < files.length; s++) { 21 | for (let t = 0; t < files.length; t++) { 22 | if (t === s) continue; 23 | 24 | target_files.push([ 25 | Path.join(__dirname, 'source', files[s]), 26 | Path.join(__dirname, 'source', files[t]), 27 | Path.join(__dirname, 'target', 28 | Path.basename(files[s], Path.extname(files[s])) + 29 | '_' + 30 | Path.basename(files[t], Path.extname(files[t])) + '.diff'), 31 | crypto.createHash('sha256').update(await fs.promises.readFile(Path.join(__dirname, 'source', files[t]))).digest('hex') 32 | ]); 33 | } 34 | } 35 | 36 | console.log('Generating diffs...'); 37 | 38 | for (let f of target_files) { 39 | let [ sourceFile, targetFile, diffFile ] = f; 40 | await spawnHelper(jdiff, [ 41 | sourceFile, 42 | targetFile, 43 | diffFile 44 | ], { 45 | acceptableExitCodes: [ 0, 1, 2 ], // all of these can be returned by jdiff 46 | }); 47 | 48 | console.log('Generated', diffFile); 49 | } 50 | 51 | console.log('\nPatching...\n'); 52 | 53 | let total_time = 0; 54 | let total_ok = 0; 55 | let total_failed = 0; 56 | 57 | for (let f of target_files) { 58 | let [ sourceFile, targetFile, diffFile, hash ] = f; 59 | let patched = diffFile.replace(/\.diff$/, '.patched'); 60 | 61 | let start = Date.now(); 62 | await spawnHelper(Path.join(__dirname, '..', 'janpatch-cli'), [ 63 | sourceFile, 64 | diffFile, 65 | patched, 66 | ]); 67 | let elapsed = '(' + ((Date.now() - start)) + ' ms.)'; 68 | 69 | total_time += (Date.now() - start); 70 | 71 | let newHash = crypto.createHash('sha256').update(await fs.promises.readFile(patched)).digest('hex'); 72 | 73 | let name = Path.basename(patched, '.patched'); 74 | 75 | if (newHash !== hash) { 76 | console.log(' \u2717', name, 'Hash mismatch (expected: ' + hash + ', but was ' + newHash + ')', elapsed); 77 | total_failed++; 78 | } 79 | else { 80 | console.log(' \u2714', name, 'OK', elapsed); 81 | total_ok++; 82 | } 83 | } 84 | 85 | console.log(`\nSummary: ${total_ok}/${total_ok+total_failed} succeeded (total time: ${total_time} ms.)`); 86 | } 87 | catch (ex) { 88 | console.error(`Failed to run integration tests`, ex); 89 | process.exit(1); 90 | } 91 | })(); 92 | 93 | function spawnHelper(command, args, opts) { 94 | opts = opts || { }; 95 | if (typeof opts.ignoreErrors !== 'boolean') { 96 | opts.ignoreErrors = false; 97 | } 98 | if (typeof opts.acceptableExitCodes === 'undefined') { 99 | opts.acceptableExitCodes = [ 0 ]; 100 | } 101 | 102 | return new Promise((resolve, reject) => { 103 | // console.log(`spawning ${command} ${args.join(' ')}`); 104 | const p = spawn(command, args, { env: process.env, cwd: opts.cwd }); 105 | 106 | const allData = []; 107 | 108 | p.stdout.on('data', (data) => { 109 | allData.push(data); 110 | // process.stdout.write(data); 111 | }); 112 | 113 | p.stderr.on('data', (data) => { 114 | allData.push(data); 115 | // process.stderr.write(data); 116 | }); 117 | 118 | p.on('error', reject); 119 | 120 | p.on('close', (code) => { 121 | if (opts.acceptableExitCodes.indexOf(code) > -1 || opts.ignoreErrors === true) { 122 | resolve(Buffer.concat(allData).toString('utf-8')); 123 | } 124 | else { 125 | let errorCodeStr = opts.acceptableExitCodes.length === 1 ? 126 | `Error code was not ${opts.acceptableExitCodes[0]}` : 127 | `Error code was not in ${JSON.stringify(opts.acceptableExitCodes)}`; 128 | reject(`${errorCodeStr} (but ${code}): ` + Buffer.concat(allData).toString('utf-8')); 129 | } 130 | }); 131 | }); 132 | } 133 | 134 | async function pathExists(path) { 135 | let exists = false; 136 | try { 137 | await util.promisify(fs.stat)(path); 138 | exists = true; 139 | } 140 | catch (ex) { 141 | /* noop */ 142 | } 143 | return exists; 144 | } 145 | -------------------------------------------------------------------------------- /integration-tests/source/00.bin: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /integration-tests/source/FF.bin: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/janjongboom/janpatch/c8ad3eeabf75640d6348a706abe7e3377a21ad87/integration-tests/source/FF.bin -------------------------------------------------------------------------------- /integration-tests/source/blinky-f411re.bin: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/janjongboom/janpatch/c8ad3eeabf75640d6348a706abe7e3377a21ad87/integration-tests/source/blinky-f411re.bin -------------------------------------------------------------------------------- /integration-tests/source/blinky-ff1705.bin: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/janjongboom/janpatch/c8ad3eeabf75640d6348a706abe7e3377a21ad87/integration-tests/source/blinky-ff1705.bin -------------------------------------------------------------------------------- /integration-tests/source/blinky-k64f.bin: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/janjongboom/janpatch/c8ad3eeabf75640d6348a706abe7e3377a21ad87/integration-tests/source/blinky-k64f.bin -------------------------------------------------------------------------------- /integration-tests/source/pg16328-images.epub: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/janjongboom/janpatch/c8ad3eeabf75640d6348a706abe7e3377a21ad87/integration-tests/source/pg16328-images.epub -------------------------------------------------------------------------------- /integration-tests/source/pg1952-images.epub: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/janjongboom/janpatch/c8ad3eeabf75640d6348a706abe7e3377a21ad87/integration-tests/source/pg1952-images.epub -------------------------------------------------------------------------------- /integration-tests/source/pg2591-images.epub: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/janjongboom/janpatch/c8ad3eeabf75640d6348a706abe7e3377a21ad87/integration-tests/source/pg2591-images.epub -------------------------------------------------------------------------------- /integration-tests/source/pg844-images.epub: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/janjongboom/janpatch/c8ad3eeabf75640d6348a706abe7e3377a21ad87/integration-tests/source/pg844-images.epub -------------------------------------------------------------------------------- /integration-tests/source/pg98-images.epub: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/janjongboom/janpatch/c8ad3eeabf75640d6348a706abe7e3377a21ad87/integration-tests/source/pg98-images.epub -------------------------------------------------------------------------------- /janpatch.h: -------------------------------------------------------------------------------- 1 | #ifndef _JANPATCH_H 2 | #define _JANPATCH_H 3 | 4 | #include 5 | #include 6 | #include 7 | 8 | #ifndef JANPATCH_DEBUG 9 | #define JANPATCH_DEBUG(...) while (0) {} // printf(__VA_ARGS__) 10 | #endif 11 | 12 | #ifndef JANPATCH_ERROR 13 | #define JANPATCH_ERROR(...) printf(__VA_ARGS__) 14 | #endif 15 | 16 | // detect POSIX, and use FILE* in that case 17 | #if !defined(JANPATCH_STREAM) && (defined (__unix__) || (defined (__APPLE__) && defined (__MACH__))) 18 | #include 19 | #define JANPATCH_STREAM FILE 20 | #elif !defined(JANPATCH_STREAM) 21 | #error "JANPATCH_STREAM not defined, and not on POSIX system. Please specify the JANPATCH_STREAM macro" 22 | #endif 23 | 24 | typedef struct { 25 | unsigned char* buffer; 26 | size_t size; 27 | uint32_t current_page; 28 | size_t current_page_size; 29 | JANPATCH_STREAM* stream; 30 | long int position; 31 | } janpatch_buffer; 32 | 33 | typedef struct { 34 | // fread/fwrite buffers 35 | janpatch_buffer source_buffer; 36 | janpatch_buffer patch_buffer; 37 | janpatch_buffer target_buffer; 38 | 39 | // function signatures 40 | size_t (*fread)(void*, size_t, size_t, JANPATCH_STREAM*); 41 | size_t (*fwrite)(const void*, size_t, size_t, JANPATCH_STREAM*); 42 | int (*fseek)(JANPATCH_STREAM*, long int, int); 43 | long (*ftell)(JANPATCH_STREAM*); 44 | 45 | // progress callback 46 | void (*progress)(uint8_t); 47 | 48 | // the combination of the size of both the source + patch files (that's the max. the target file can be) 49 | long max_file_size; 50 | } janpatch_ctx; 51 | 52 | enum { 53 | JANPATCH_OPERATION_ESC = 0xa7, 54 | JANPATCH_OPERATION_MOD = 0xa6, 55 | JANPATCH_OPERATION_INS = 0xa5, 56 | JANPATCH_OPERATION_DEL = 0xa4, 57 | JANPATCH_OPERATION_EQL = 0xa3, 58 | JANPATCH_OPERATION_BKT = 0xa2 59 | }; 60 | 61 | /** 62 | * Read a buffer off the stream 63 | */ 64 | static size_t jp_fread(janpatch_ctx *ctx, void *ptr, size_t size, size_t count, janpatch_buffer *buffer) { 65 | ctx->fseek(buffer->stream, buffer->position, SEEK_SET); 66 | 67 | size_t bytes_read = ctx->fread(ptr, size, count, buffer->stream); 68 | 69 | buffer->position += bytes_read; 70 | 71 | return bytes_read; 72 | } 73 | 74 | /** 75 | * Write a buffer to the stream 76 | */ 77 | static size_t jp_fwrite(janpatch_ctx *ctx, const void *ptr, size_t size, size_t count, janpatch_buffer *buffer) { 78 | ctx->fseek(buffer->stream, buffer->position, SEEK_SET); 79 | 80 | size_t bytes_written = ctx->fwrite(ptr, size, count, buffer->stream); 81 | 82 | buffer->position += bytes_written; 83 | 84 | return bytes_written; 85 | } 86 | 87 | /** 88 | * Set position of the stream 89 | */ 90 | static int jp_fseek(janpatch_buffer *buffer, long int offset, int origin) { 91 | if (origin == SEEK_SET) { 92 | buffer->position = offset; 93 | } 94 | else if (origin == SEEK_CUR) { 95 | buffer->position += offset; 96 | } 97 | else { 98 | JANPATCH_ERROR("Origin %d not supported in jp_fseek (only SEEK_CUR,SEEK_SET)\n", origin); 99 | return -1; 100 | } 101 | return 0; 102 | } 103 | 104 | 105 | /** 106 | * Get a character from the stream 107 | */ 108 | static int jp_getc(janpatch_ctx* ctx, janpatch_buffer* buffer) { 109 | long position = buffer->position; 110 | if (position < 0) return -1; 111 | 112 | // calculate the current page... 113 | uint32_t page = ((unsigned long)position) / buffer->size; 114 | 115 | if (page != buffer->current_page) { 116 | jp_fseek(buffer, page * buffer->size, SEEK_SET); 117 | buffer->current_page_size = jp_fread(ctx, buffer->buffer, 1, buffer->size, buffer); 118 | buffer->current_page = page; 119 | } 120 | 121 | int position_in_page = position % buffer->size; 122 | 123 | if ((size_t)position_in_page >= buffer->current_page_size) { 124 | return EOF; 125 | } 126 | 127 | unsigned char b = buffer->buffer[position_in_page]; 128 | jp_fseek(buffer, position + 1, SEEK_SET); 129 | return b; 130 | } 131 | 132 | /** 133 | * Write a character to a stream 134 | */ 135 | static int jp_putc(int c, janpatch_ctx* ctx, janpatch_buffer* buffer) { 136 | long position = buffer->position; 137 | if (position < 0) { 138 | return -1; 139 | } 140 | 141 | // calculate the current page... 142 | uint32_t page = ((unsigned long)position) / buffer->size; 143 | 144 | if (page != buffer->current_page) { 145 | // flush the page buffer... 146 | if (buffer->current_page != 0xFFFFFFFF) { 147 | jp_fseek(buffer, buffer->current_page * buffer->size, SEEK_SET); 148 | jp_fwrite(ctx, buffer->buffer, 1, buffer->current_page_size, buffer); 149 | 150 | if (ctx->progress) { 151 | ctx->progress(position * 100 / ctx->max_file_size); 152 | } 153 | } 154 | 155 | // and read the next page... 156 | jp_fseek(buffer, page * buffer->size, SEEK_SET); 157 | jp_fread(ctx, buffer->buffer, 1, buffer->size, buffer); 158 | buffer->current_page_size = buffer->size; 159 | buffer->current_page = page; 160 | } 161 | 162 | int position_in_page = position % buffer->size; 163 | 164 | buffer->buffer[position_in_page] = (unsigned char)c; 165 | jp_fseek(buffer, position + 1, SEEK_SET); 166 | 167 | return 0; 168 | } 169 | 170 | static void jp_final_flush(janpatch_ctx* ctx, janpatch_buffer* buffer) { 171 | long position = buffer->position; 172 | int position_in_page = position % buffer->size; 173 | 174 | uint32_t page = ((unsigned long)position) / buffer->size; 175 | 176 | // if the page has changed we also need to flush the previous page 177 | // this can happen when the last operation (e.g. jp_putc) has just crossed page boundary 178 | if (page != buffer->current_page) { 179 | // flush the page buffer... 180 | if (buffer->current_page != 0xFFFFFFFF) { 181 | jp_fseek(buffer, buffer->current_page * buffer->size, SEEK_SET); 182 | jp_fwrite(ctx, buffer->buffer, 1, buffer->current_page_size, buffer); 183 | } 184 | } 185 | 186 | // flush the new page buffer 187 | jp_fseek(buffer, page * buffer->size, SEEK_SET); 188 | jp_fwrite(ctx, buffer->buffer, 1, position_in_page, buffer); 189 | 190 | if (ctx->progress) { 191 | ctx->progress(100); 192 | } 193 | } 194 | 195 | static void process_mod(janpatch_ctx *ctx, janpatch_buffer *source, janpatch_buffer *patch, janpatch_buffer *target, bool up_source_stream) { 196 | // it can be that ESC character is actually in the data, but then it's prefixed with another ESC 197 | // so... we're looking for a lone ESC character 198 | size_t cnt = 0; 199 | while (1) { 200 | cnt++; 201 | int m = jp_getc(ctx, patch); 202 | if (m == -1) { 203 | // End of file stream... rewind 1 character and return, this will yield back to janpatch main function, which will exit 204 | jp_fseek(source, -1, SEEK_CUR); 205 | return; 206 | } 207 | // JANPATCH_DEBUG("%02x ", m); 208 | // so... if it's *NOT* an ESC character, just write it to the target stream 209 | if (m != JANPATCH_OPERATION_ESC) { 210 | // JANPATCH_DEBUG("NOT ESC\n"); 211 | jp_putc(m, ctx, target); 212 | if (up_source_stream) { 213 | jp_fseek(source, 1, SEEK_CUR); // and up source 214 | } 215 | continue; 216 | } 217 | 218 | // read the next character to see what we should do 219 | m = jp_getc(ctx, patch); 220 | // JANPATCH_DEBUG("%02x ", m); 221 | 222 | if (m == -1) { 223 | // End of file stream... rewind 1 character and return, this will yield back to janpatch main function, which will exit 224 | jp_fseek(source, -1, SEEK_CUR); 225 | return; 226 | } 227 | 228 | // if the character after this is *not* an operator (except ESC) 229 | if (m == JANPATCH_OPERATION_ESC) { 230 | // JANPATCH_DEBUG("ESC, NEXT CHAR ALSO ESC\n"); 231 | jp_putc(m, ctx, target); 232 | if (up_source_stream) { 233 | jp_fseek(source, 1, SEEK_CUR); 234 | } 235 | } 236 | else if (m >= 0xA2 && m <= 0xA6) { // character after this is an operator? Then roll back two characters and exit 237 | // JANPATCH_DEBUG("ESC, THEN OPERATOR\n"); 238 | JANPATCH_DEBUG("%lu bytes\n", cnt); 239 | jp_fseek(patch, -2, SEEK_CUR); 240 | break; 241 | } 242 | else { // else... write both the ESC and m 243 | // JANPATCH_DEBUG("ESC, BUT NO OPERATOR\n"); 244 | jp_putc(JANPATCH_OPERATION_ESC, ctx, target); 245 | jp_putc(m, ctx, target); 246 | if (up_source_stream) { 247 | jp_fseek(source, 2, SEEK_CUR); // up source by 2 248 | } 249 | } 250 | } 251 | } 252 | 253 | static int find_length(janpatch_ctx *ctx, janpatch_buffer *buffer) { 254 | /* So... the EQL/BKT length thing works like this: 255 | * 256 | * If byte[0] is between 1..251 => use byte[0] + 1 257 | * If byte[0] is 252 => use ??? 258 | * If byte[0] is 253 => use (byte[1] << 8) + byte[2] 259 | * If byte[0] is 254 => use (byte[1] << 16) + (byte[2] << 8) + byte[3] (NOT VERIFIED) 260 | */ 261 | uint8_t l = jp_getc(ctx, buffer); 262 | if (l <= 251) { 263 | return l + 1; 264 | } 265 | else if (l == 252) { 266 | return l + jp_getc(ctx, buffer) + 1; 267 | } 268 | else if (l == 253) { 269 | return (jp_getc(ctx, buffer) << 8) + jp_getc(ctx, buffer); 270 | } 271 | else if (l == 254) { 272 | return (jp_getc(ctx, buffer) << 24) + (jp_getc(ctx, buffer) << 16) + (jp_getc(ctx, buffer) << 8) + (jp_getc(ctx, buffer)); 273 | } 274 | else { 275 | JANPATCH_ERROR("EQL followed by unexpected byte %02x %02x\n", JANPATCH_OPERATION_EQL, l); 276 | return -1; 277 | } 278 | 279 | // it's fine if we get over the end of the stream here, will be caught by the next function 280 | } 281 | 282 | int janpatch(janpatch_ctx ctx, JANPATCH_STREAM *source, JANPATCH_STREAM *patch, JANPATCH_STREAM *target) { 283 | 284 | ctx.source_buffer.current_page = 0xffffffff; 285 | ctx.patch_buffer.current_page = 0xffffffff; 286 | ctx.target_buffer.current_page = 0xffffffff; 287 | 288 | ctx.source_buffer.position = 0; 289 | ctx.patch_buffer.position = 0; 290 | ctx.target_buffer.position = 0; 291 | 292 | ctx.source_buffer.stream = source; 293 | ctx.patch_buffer.stream = patch; 294 | ctx.target_buffer.stream = target; 295 | 296 | // look at the size of the source file... 297 | if (ctx.progress != NULL && ctx.ftell != NULL) { 298 | ctx.fseek(source, 0, SEEK_END); 299 | ctx.max_file_size = ctx.ftell(source); 300 | JANPATCH_DEBUG("Source file size is %ld\n", ctx.max_file_size); 301 | ctx.fseek(source, 0, SEEK_SET); 302 | 303 | // and at the size of the patch file 304 | ctx.fseek(patch, 0, SEEK_END); 305 | ctx.max_file_size += ctx.ftell(patch); 306 | JANPATCH_DEBUG("Now max file size is %ld\n", ctx.max_file_size); 307 | ctx.fseek(patch, 0, SEEK_SET); 308 | } 309 | else { 310 | ctx.progress = NULL; 311 | } 312 | 313 | int c; 314 | while ((c = jp_getc(&ctx, &ctx.patch_buffer)) != EOF) { 315 | if (c == JANPATCH_OPERATION_ESC) { 316 | c = jp_getc(&ctx, &ctx.patch_buffer); 317 | } 318 | else { 319 | // Starting from JoJoDiff v0.8.5, the default operation is MOD. 320 | // The currently read character from the patch stream is not an ESC, 321 | // therefore rewind 1 character in the patch stream and set the 322 | // default operation to MOD in order to process that character. 323 | jp_fseek(&ctx.patch_buffer, -1, SEEK_CUR); 324 | c = JANPATCH_OPERATION_MOD; 325 | } 326 | 327 | switch (c) { 328 | case JANPATCH_OPERATION_EQL: { 329 | int length = find_length(&ctx, &ctx.patch_buffer); 330 | if (length == -1) { 331 | JANPATCH_ERROR("EQL length invalid\n"); 332 | JANPATCH_ERROR("Positions are, source=%ld patch=%ld new=%ld\n", ctx.source_buffer.position, ctx.patch_buffer.position, ctx.target_buffer.position); 333 | return 1; 334 | } 335 | 336 | JANPATCH_DEBUG("EQL: %d bytes\n", length); 337 | 338 | for (int ix = 0; ix < length; ix++) { 339 | int r = jp_getc(&ctx, &ctx.source_buffer); 340 | if (r < -1) { 341 | JANPATCH_ERROR("fread returned %d, but expected character\n", r); 342 | JANPATCH_ERROR("Positions are, source=%ld patch=%ld new=%ld\n", ctx.source_buffer.position, ctx.patch_buffer.position, ctx.target_buffer.position); 343 | return 1; 344 | } 345 | 346 | jp_putc(r, &ctx, &ctx.target_buffer); 347 | } 348 | 349 | break; 350 | } 351 | case JANPATCH_OPERATION_MOD: { 352 | JANPATCH_DEBUG("MOD: "); 353 | 354 | // MOD means to modify the next series of bytes 355 | // so just write everything (until the next ESC sequence) to the target JANPATCH_STREAM 356 | // but also up the position in the source JANPATCH_STREAM every time 357 | process_mod(&ctx, &ctx.source_buffer, &ctx.patch_buffer, &ctx.target_buffer, true); 358 | break; 359 | } 360 | case JANPATCH_OPERATION_INS: { 361 | JANPATCH_DEBUG("INS: "); 362 | // INS inserts the sequence in the new JANPATCH_STREAM, but does not up the position of the source JANPATCH_STREAM 363 | // so just write everything (until the next ESC sequence) to the target JANPATCH_STREAM 364 | 365 | process_mod(&ctx, &ctx.source_buffer, &ctx.patch_buffer, &ctx.target_buffer, false); 366 | break; 367 | } 368 | case JANPATCH_OPERATION_BKT: { 369 | // BKT = backtrace, seek back in source JANPATCH_STREAM with X bytes... 370 | int length = find_length(&ctx, &ctx.patch_buffer); 371 | if (length == -1) { 372 | JANPATCH_ERROR("BKT length invalid\n"); 373 | JANPATCH_ERROR("Positions are, source=%ld patch=%ld new=%ld\n", ctx.source_buffer.position, ctx.patch_buffer.position, ctx.target_buffer.position); 374 | return 1; 375 | } 376 | 377 | JANPATCH_DEBUG("BKT: %d bytes\n", -length); 378 | 379 | jp_fseek(&ctx.source_buffer, -length, SEEK_CUR); 380 | 381 | break; 382 | } 383 | case JANPATCH_OPERATION_DEL: { 384 | // DEL deletes bytes, so up the source stream with X bytes 385 | int length = find_length(&ctx, &ctx.patch_buffer); 386 | if (length == -1) { 387 | JANPATCH_ERROR("DEL length invalid\n"); 388 | JANPATCH_ERROR("Positions are, source=%ld patch=%ld new=%ld\n", ctx.source_buffer.position, ctx.patch_buffer.position, ctx.target_buffer.position); 389 | return 1; 390 | } 391 | 392 | JANPATCH_DEBUG("DEL: %d bytes\n", length); 393 | 394 | jp_fseek(&ctx.source_buffer, length, SEEK_CUR); 395 | break; 396 | } 397 | case -1: { 398 | // End of file stream... rewind 1 character and break, this will yield back to main loop 399 | jp_fseek(&ctx.source_buffer, -1, SEEK_CUR); 400 | break; 401 | } 402 | default: { 403 | // Breaking change in JoJoDiff v0.8.5: 404 | // The ESC-MOD sequence is now the default when a new operation 405 | // sequence is needed (at the start of a file or after an EQL, 406 | // DEL or BKT operation). 407 | // See: https://github.com/janjongboom/janpatch/issues/16#issuecomment-705010958 408 | jp_fseek(&ctx.patch_buffer, -2, SEEK_CUR); 409 | process_mod(&ctx, &ctx.source_buffer, &ctx.patch_buffer, &ctx.target_buffer, true); 410 | break; 411 | } 412 | } 413 | } 414 | 415 | jp_final_flush(&ctx, &ctx.target_buffer); 416 | 417 | return 0; 418 | } 419 | 420 | #endif // _JANPATCH_H 421 | -------------------------------------------------------------------------------- /jdiff-binary-sources/.gitignore: -------------------------------------------------------------------------------- 1 | jdiff07/ 2 | jdiff081/ 3 | jdiff085/ 4 | -------------------------------------------------------------------------------- /jdiff-binary-sources/build-all.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | SCRIPTPATH="$( cd "$(dirname "$0")" ; pwd -P )" 5 | 6 | if [[ "$OSTYPE" == "darwin"* ]]; then 7 | SEDCMD="sed -i '' -e" 8 | LC_CTYPE=C 9 | LANG=C 10 | else 11 | SEDCMD="sed -i -e" 12 | fi 13 | 14 | rm -rf $SCRIPTPATH/jdiff07/ 15 | rm -rf $SCRIPTPATH/jdiff081/ 16 | rm -rf $SCRIPTPATH/jdiff085/ 17 | 18 | echo "Extracting jdiff07..." 19 | cd $SCRIPTPATH 20 | rm -rf jdiff07 21 | mkdir -p jdiff07 22 | tar -xzf jdiff07.tgz -C jdiff07 23 | mv jdiff07/jojodiff07/* jdiff07 24 | rm -r jdiff07/jojodiff07/ 25 | echo "Extracting jdiff07 OK" 26 | echo "" 27 | 28 | echo "Extracting jdiff081..." 29 | cd $SCRIPTPATH 30 | rm -rf jdiff081 31 | mkdir -p jdiff081 32 | tar -xzf jdiff081.tgz -C jdiff081 33 | mv jdiff081/jdiff081/* jdiff081 34 | rm -r jdiff081/jdiff081/ 35 | echo "Extracting jdiff081 OK" 36 | echo "" 37 | 38 | echo "Extracting jdiff085..." 39 | cd $SCRIPTPATH 40 | rm -rf jdiff085 41 | mkdir -p jdiff085 42 | tar -xzf jdiff085.tgz -C jdiff085 43 | mv jdiff085/jdiff085/* jdiff085 44 | rm -r jdiff085/jdiff085/ 45 | echo "Extracting jdiff085 OK" 46 | echo "" 47 | 48 | echo "Building jdiff07..." 49 | cd $SCRIPTPATH/jdiff07/src 50 | $SEDCMD "s/GCC4=gcc-4/GCC4=gcc/g" Makefile # patch out gcc-4 51 | make -j`nproc` 52 | mv jdiff.exe jdiff 53 | if [ -f "Makefile''" ]; then 54 | rm "Makefile''" 55 | fi 56 | echo "Building jdiff07 OK" 57 | echo "" 58 | 59 | echo "Building jdiff081..." 60 | cd $SCRIPTPATH/jdiff081/src 61 | $SEDCMD "s/#define ulong unsigned long int//g" JDefs.h # patch out unused ulong define 62 | if [ -f "JDefs.h''" ]; then 63 | rm "JDefs.h''" 64 | fi 65 | make -j`nproc` 66 | echo "Building jdiff081 OK" 67 | echo "" 68 | 69 | echo "Building jdiff085..." 70 | cd $SCRIPTPATH/jdiff085/src 71 | $SEDCMD "s/-m64//g" Makefile # patch out -m64 72 | if [ -f "Makefile''" ]; then 73 | rm "Makefile''" 74 | fi 75 | make -j`nproc` 76 | echo "Building jdiff085 OK" 77 | echo "" 78 | -------------------------------------------------------------------------------- /jdiff-binary-sources/jdiff07.tgz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/janjongboom/janpatch/c8ad3eeabf75640d6348a706abe7e3377a21ad87/jdiff-binary-sources/jdiff07.tgz -------------------------------------------------------------------------------- /jdiff-binary-sources/jdiff081.tgz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/janjongboom/janpatch/c8ad3eeabf75640d6348a706abe7e3377a21ad87/jdiff-binary-sources/jdiff081.tgz -------------------------------------------------------------------------------- /jdiff-binary-sources/jdiff085.tgz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/janjongboom/janpatch/c8ad3eeabf75640d6348a706abe7e3377a21ad87/jdiff-binary-sources/jdiff085.tgz -------------------------------------------------------------------------------- /jdiff-binary-sources/source.txt: -------------------------------------------------------------------------------- 1 | Downloaded from https://sourceforge.net/projects/jojodiff/files/jojodiff/ - mirroring in this repo. 2 | --------------------------------------------------------------------------------