├── LICENSE ├── README.md ├── asm.py ├── avg.py ├── avg_pico.py ├── avgtest.py ├── avgtest_pico.py ├── coeff_format.py ├── fir.py ├── fir_py.py ├── firtest.py ├── images ├── lpf_bode.jpg └── lpf_nyquist.jpg ├── lpf.py ├── non_realtime ├── FILT.md ├── autocorrelate.py ├── coeffs.py ├── correlate.jpg ├── correlate.py ├── filt.py ├── filt_test.py ├── filt_test_all.py └── samples.py └── osc.py /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 Peter Hinch 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Fast Filters for the Pyboard 2 | 3 | V0.92 20th December 2021 Updated to improve portability. 4 | Author: Peter Hinch 5 | 6 | # Introduction 7 | 8 | This repository is intended for high speed filtering of integer data acquired 9 | from transducers. For those unfamiliar with digital filtering, please see the 10 | last section of this doc. 11 | 12 | The repository comprises two sections. 13 | 14 | ## Realtime filtering 15 | 16 | This handles a continuous stream of samples and conceptually works as follows 17 | (pseudocode): 18 | ```python 19 | filt = MyFilter(args) 20 | while True: 21 | s = MyTransducer.get() # Acquire an integer sample 22 | t = filt(s) # Filter it 23 | # Use the filtered result 24 | ``` 25 | Support is provided for moving average and FIR (finite impulse response) 26 | filtering. FIR filters may be defined using a web application to provide low 27 | pass, high pass or bandpass characteristics. 28 | 29 | The `fir.py` and `avg.py` modules use ARM Thumb V6 inline assembly language for 30 | performance and can run on Pyboards (Thumb V7) and also the Raspberry Pico 31 | (V6). The `fir_py.py` module is written in pure Python using the Viper emitter 32 | for performance. It is therefore portable, but runs at about 33% of the speed 33 | of the assembler version. 34 | 35 | Filter functions may be called from hard ISR's. 36 | 37 | The following images show Bode and Nyquist plots of measured results from a 38 | Pyboard 1.1. The test signal was fed into a Pyboard ADC, with the resultant 39 | signal on the DAC being plotted. -60dB is the noise floor of my home-brew 40 | network analyser. 41 | 42 | ![Image](./images/lpf_bode.jpg) 43 | 44 | ![Image](./images/lpf_nyquist.jpg) 45 | 46 | ## Non-realtime filtering 47 | 48 | This processes a set of samples in a buffer, for example processing sample sets 49 | acquired by `ADC.read_timed()`. It requires the ARMV7 assembler and is 50 | therefore restricted to Pyboard and similar targets. The files and docs are in 51 | the `non_realtime` directory. See [the docs](./non_realtime/FILT.md). 52 | 53 | The algorithm can be configured for continuous (circular buffer) or 54 | discontinuous sample sets. It can optionally perform decimation. In addition to 55 | FIR filtering it can be employed for related functions such as convolution, 56 | cross- and auto-correlation. 57 | 58 | The remainder of this README describes the realtime options. 59 | 60 | # Designing the filter 61 | 62 | The first stage in designing a filter is to determine the coefficients. FIR 63 | filters can be designed for low pass, high pass, bandpass or band stop 64 | applications and the site [TFilter](http://t-filter.engineerjs.com/) enables 65 | these to be computed. I have provided a utility `coeff_format.py` to simplify 66 | the conversion of coefficients into Python code. Use the website cited above 67 | and set it to provide integer coefficients. Cut and paste the list of 68 | coefficients into a file: this will have one integer per line. Then run: 69 | ```bash 70 | python3 coeff_format.py inputfilename outputfilename.py 71 | ``` 72 | The result will be Python code defining the array. 73 | 74 | ## Coefficient order 75 | 76 | The website cited above generates symmetrical (linear phase) sets of 77 | coefficients. In other words, for a set of n coefficients, 78 | `coeff[x] == coeff[n-x]`. For coefficient arrays lacking this symmetry note 79 | that the code applies coefficients to samples such that the oldest sample is 80 | multiplied by `coeff[0]` and so on with the newest getting `coeff[n]`. 81 | 82 | ## Scaling 83 | 84 | Calculations are based on 32 bit signed arithmetic. Given that few transducers 85 | offer more than 16 bit precision there is a lot of headroom. Nevertheless 86 | overflow can occur depending on the coefficient size. The maximum output from 87 | the multiplication is `max(data)*max(coeffs)` but the subsequent addition offers 88 | further scope for overflow. In applications with analog output there is little 89 | point in creating results with more precision than the output DAC. The `fir()` 90 | function includes scaling by performing an arithmetic right shift on the result 91 | of each multiplication. This can be in the range of 0-31 bits, although 20 bits 92 | is a typical maximum. 93 | For an analytical way to determine the minimum scaling required to prevent 94 | overflow see Appendix 1. The `lpf.py` example applies a scaling of 16 bits to 95 | preserve the 12 bit resolution of the ADC which is then scaled in Python to 96 | match the DAC. 97 | 98 | ## Solutions 99 | 100 | Two MicroPython solutions are offered. The choice depends on the platform in 101 | use. `fir_py.py` uses the Viper code emitter and should run on any platform. 102 | `fir.py` uses inline Arm Thumb assembler and will run on hosts using ARM V6 or 103 | later. This includes all Pyboards and boards using the Raspberry RP2 chip (e.g. 104 | the Raspberry Pico). Using the Assembler version is slightly more inolved but 105 | it runs about three times faster (on the order of 15μs). 106 | 107 | ## Portable FIR using Viper 108 | 109 | The `fir_py` module uses a closure to enable the function to retain state 110 | between calls. Usage is as follows: 111 | ```python 112 | from fir_py import create_fir 113 | from array import array 114 | # 21 tap LPF. Figures from TFilter. 115 | coeffs = array('i', (-1318, -3829, -4009, -717, 3359, 2177, -3706, -5613, 116 | 4154, 20372, 28471, 20372, 4154, -5613, -3706, 2177, 117 | 3359, -717, -4009, -3829, -1318)) 118 | fir = create_fir(coeffs, 0) # Instantiate fir function. No scaling. 119 | 120 | print(fir(1)) 121 | for n in range(len(coeffs)+3): 122 | print(fir(0)) 123 | ``` 124 | This example simulates an impulse function passing through the filter. The 125 | outcome simply replays the coefficients followed by zeros once the impulse has 126 | cleared the filter. 127 | 128 | The `create_fir` function takes the following mandatory positional args: 129 | 1. `coeffs` A 32 bit integer array of coefficients. 130 | 2. `shift` The result of each multiplication is shifted right by `shift` bits 131 | before adding to the result. See Scaling above. 132 | 133 | Note that Viper can issue very confusing error messages. If these occur, check 134 | the data types passed to `create_fir` and `fir`. 135 | 136 | ## FIR using ARM Thumb Assembler 137 | 138 | In addition to the coefficient array the Assembler version requires the user to 139 | pass an array to hold the set of samples 140 | 141 | The `fir.fir()` function takes three arguments: 142 | 1. An integer array of length equal to the number of coeffcients + 3. 143 | 2. An integer array of coefficients. 144 | 3. The new data value. 145 | 146 | The function returns an integer which is the current filtered value. 147 | The array must be initialised as follows: 148 | 1. `data[0]` should be set to len(data). 149 | 2. `data[1]` is a scaling value in range 0..31: see scaling above. 150 | 3. Other elements of the data array must be zero. 151 | 152 | Usage is along these lines: 153 | ```python 154 | from fir import fir 155 | from array import array 156 | # 21 tap LPF. Figures from TFilter. 157 | coeffs = array('i', (-1318, -3829, -4009, -717, 3359, 2177, -3706, -5613, 158 | 4154, 20372, 28471, 20372, 4154, -5613, -3706, 2177, 159 | 3359, -717, -4009, -3829, -1318)) 160 | ncoeffs = len(coeffs) 161 | data = array('i', (0 for _ in range(ncoeffs + 3))) 162 | data[0] = ncoeffs 163 | data[1] = 0 # No scaling 164 | 165 | print(fir(data, coeffs, 1)) 166 | for n in range(ncoeffs + 3): 167 | print(fir(data, coeffs, 0)) 168 | ``` 169 | This example simulates an impulse function passing through the filter. The 170 | outcome simply replays the coefficients followed by zeros once the impulse has 171 | cleared the filter. 172 | 173 | ## Demo program 174 | 175 | The file `lpf.py` uses a Pyboard as a low pass filter with a cutoff of 40Hz. It 176 | processes an analog input presented on pin X7, filters it, and outputs the 177 | result on DAC2 (X6). For convenience the code includes a swept frequency 178 | oscillator with output on DAC1 (X5). By linking X5 and X7 the filtered result 179 | can be viewed on X6. 180 | 181 | The filter uses Timer 4 to sample the incoming data at 2KHz. 182 | The program generates a swept frequency sine wave on DAC1 and reads it using 183 | the ADC on pin X7. The filtered signal is output on DAC2. The incoming signal 184 | is sampled at 2KHz by means of Timer 4, with the FIR filter operating in the 185 | timer's callback handler. 186 | 187 | When using the oscillator to test filters you may see occasional transients 188 | occurring in the stopband. These are a consequence of transient frequency 189 | components caused by the step changes in the oscillator frequency: this can be 190 | demonstrated by increasing the delay between frequency changes. Ideally the 191 | oscillator would issue a slow, continuous sweep. 192 | 193 | firtest.py illustrates the FIR operation and computes execution times with 194 | different sets of coefficients. 195 | 196 | ## Performance 197 | 198 | These results were measured on a Pyboard 1.1 running firmware V1.17. Times are 199 | in μs and were measured using `firtest.py` (adapted to run the Viper version). 200 | The accuracy of these timings is suspect as they varied between runs - and it 201 | makes no sense for the 41 tap filter to run faster than the 21 tap. However 202 | they give an indication of performance. 203 | 204 | | Taps | Asm | Viper | 205 | |:----:|:---:|:-----:| 206 | | 21 | 18 | 33 | 207 | | 41 | 9 | 32 | 208 | | 109 | 30 | 93 | 209 | 210 | # Moving average 211 | 212 | A moving average is a degenerate case of an FIR filter with unity coefficients. 213 | As such it can run faster. On the Pyboard 1.1 the moving average takes about 214 | 8μs for a typical set of coefficients. 215 | 216 | The Raspberry Pico ARM V6 assembler doesn't support integer division. A special 217 | version `avg_pico.py` runs on the Pico. This offers scaling using a right shift 218 | operation which produces correct resullts if the number of entries is a power 219 | of 2. Alternatively with a scaling factor of 0 the result is `N*average` where 220 | `N` is the number of entries. 221 | 222 | On Pyboards and other ARMV7 targets, the file `avg.py` produces expected values 223 | for all `N`. Both versions provide a function `avg`. 224 | 225 | ## Moving Average Usage 226 | 227 | The `avg` function takes two arguments, or three in the Pico case: 228 | 1. An integer array of length equal to the no. of entries to average +3. 229 | 2. The new data value. 230 | 3. The scaling value (number of bits to shift right) - Pico only. 231 | 232 | The function returns an integer which is the current filtered value. 233 | Initially all elements of the data array must be zero, except `data[0]` which 234 | should be set to `len(data)` 235 | The test scripts `avgtest.py` and `avgtest_pico.py` illustrate its operation. 236 | ```python 237 | from avg_pico import avg 238 | ``` 239 | 240 | # Absolute Beginners 241 | 242 | Data arriving from transducers often needs to be filtered to render it useful. 243 | Reasons include reducing noise (random perturbations) in the data, isolating a 244 | particular signal or shaping the response to sudden changes in the data value. 245 | A common approach to reducing noise is to take a moving average of the last N 246 | samples. While this is computationally simple and hence fast, it is a 247 | relatively crude form of filtering because the oldest sample in the set has the 248 | same weight as the most recent. This is often non-optimal. 249 | 250 | FIR (finite impulse response) filters can be viewed as an extension of the 251 | moving average concept where each sample is allocated a different weight 252 | depending on its age. These weights are defined by a set of coefficients. The 253 | result is calculated by multiplying each sample by its coefficient before 254 | adding them; a moving average corresponds to the situation where all 255 | coefficients are set to 1. By adjusting the coefficients you can alter the 256 | relative weights of the data values, with the most recent having a different 257 | weight to the next most recent, and so on. 258 | 259 | In practice FIR filters can be designed to produce a range of filter types: 260 | low pass, high pass, bandpass, band stop and so on. They can also be tailored 261 | to produce a specific response to sudden changes (impulse response). The 262 | process of computing the coefficients is complex, but the link above provides a 263 | simple GUI approach. Set the application to produce 16 or 32 bit integer 264 | values, set your desired characteristics and press "Design Filter". Then 265 | proceed as suggested above to convert the results to Python code. 266 | 267 | The term "finite impulse response" describes the response of a filter to a 268 | brief (one sample) pulse. In an FIR filter with N coefficients the response 269 | drops to zero after N samples because the impulse has passed through the 270 | filter. This contrasts with an IIR (infinite impulse response) filter where the 271 | response theoretically continues forever. Analog circuits such as a CR network 272 | can have an IIR response, as do some digital filters. 273 | 274 | # Appendix 1 Determining scaling 275 | 276 | The following calculation determines the number of bits required to represent 277 | the outcome when a worst-case signal passes through an FIR filter. The ADC is 278 | assumed to be biassed for symmetrical output. The worst-case signal, at some 279 | amount of shift through the filter, has maximum positive excursion matching 280 | positive coefficients and maximum negative excursion matching negative 281 | coefficients. This will produce the largest possible positive sum. By symmetry 282 | a negative result of equal magnitude could result, where a negative signal 283 | matches a positive coefficient and vice-versa. 284 | 285 | There are two places where overflow can occur: in the multiplication and in the 286 | subsequent addition. The former cannot be compensated: the coefficients need to 287 | be reduced in size. The latter can be compensated by performing a right shift 288 | after the multiplication, and the Assembler routine provides for this. 289 | 290 | The following code calculates the number of bits required to accommodate this 291 | result. On 32-bit platforms, small integers occupy 31 bits holding values up to 292 | +-2^30. Consequently if this script indicates that 33 bits are required, 293 | scaling of at least 2 bits must be applied to guarantee no overflow. 294 | 295 | ```python 296 | from math import log 297 | 298 | # Return no. of bits to contain a positive integer 299 | def nbits(n : int) -> int: 300 | return int(log(n) // log(2)) + 1 301 | 302 | def get_shift(coeffs : list, adcbits : int =12): 303 | # Assume ADC is biassed for equal + and - excursions 304 | maxadc = (2 ** (adcbits - 1) - 1) # 2047 for 12 bit ADC 305 | lv = sorted([abs(x) * maxadc for x in coeffs], reverse=True) 306 | # Add 1 to allow for equal negative swing 307 | print("Max no. of bits for multiply", nbits(lv[0]) + 1) 308 | print("Max no. of bits for sum of products", nbits(sum(lv) + 1)) 309 | ``` 310 | 311 | 312 | -------------------------------------------------------------------------------- /asm.py: -------------------------------------------------------------------------------- 1 | # Examples of the workround for unimplemented assembler instructions 2 | # Author: Peter Hinch 3 | # 15th Feb 2015 4 | # Note: these instructions are now supported. I've left this file in place 5 | # to illustrate the technique. 6 | 7 | # Source: ARM v7-M Architecture Reference Manual 8 | 9 | # Make r8-r11 safe to use by pushing them on the stack 10 | @micropython.asm_thumb 11 | def foo(r0): 12 | data(2, 0xe92d, 0x0f00) # push r8,r9,r10,r11 13 | mov(r8, r0) # Would otherwise crash the board! 14 | data(2, 0xe8bd, 0x0f00) # pop r8,r9,r10,r11 15 | 16 | # The signed divide instruction 17 | @micropython.asm_thumb 18 | def div(r0, r1): 19 | data(2, 0xfb90, 0xf0f1) 20 | 21 | # Bit reversal 22 | @micropython.asm_thumb 23 | def rbit(r0): 24 | data(2, 0xfa90, 0xf0a0) 25 | 26 | # Count leading zeros 27 | @micropython.asm_thumb 28 | def clz(r0): 29 | data(2, 0xfab0, 0xf080) 30 | 31 | # Count trailing zeros 32 | @micropython.asm_thumb 33 | def clz(r0): 34 | data(2, 0xfa90, 0xf0a0) # Bit reverse 35 | data(2, 0xfab0, 0xf080) # count leading zeros 36 | 37 | -------------------------------------------------------------------------------- /avg.py: -------------------------------------------------------------------------------- 1 | # Implementation of moving average filter in Arm Thumb assembler 2 | # Released under the MIT License (MIT). See LICENSE. 3 | # Copyright (c) 2021 Peter Hinch 4 | # Timing: 27uS on MicroPython board (independent of data) 5 | 6 | # Function arguments: 7 | # r0 is an integer scratchpad array. Must be of length 3 greater than 8 | # the number of values to be averaged. 9 | # On entry array[0] must hold the array length, other elements must be zero 10 | # r1 holds new data value 11 | 12 | # Return value: the current moving average 13 | 14 | # array[0] is array length, array[1] is the current sum, array[2] the insertion point 15 | # r2 holds the length of the coefficient array 16 | # Pointers (byte addresses) 17 | # r3 start of ring buffer 18 | # r4 insertion point (post increment) 19 | # r5 last location of ring buffer 20 | # Other registers 21 | # r7 temporary store for result 22 | 23 | @micropython.asm_thumb 24 | def avg(r0, r1): 25 | mov(r3, r0) 26 | add(r3, 12) # r3 points to ring buffer start 27 | ldr(r2, [r0, 0]) # Element count 28 | sub(r2, 4) # Offset in words to buffer end 29 | add(r2, r2, r2) 30 | add(r2, r2, r2) # convert to bytes 31 | add(r5, r2, r3) # r5 points to ring buffer end (last valid address) 32 | ldr(r4, [r0, 8]) # Current insertion point address 33 | cmp(r4, 0) # If it's zero we need to initialise 34 | bne(INIT) 35 | mov(r4, r3) # Initialise: point to buffer start 36 | label(INIT) 37 | ldr(r7, [r0, 4]) # get current sum 38 | ldr(r6, [r4, 0]) 39 | sub(r7, r7, r6) # deduct oldest value 40 | add(r7, r7, r1) # add newest value 41 | str(r7, [r0, 4]) # put sum back 42 | str(r1, [r4, 0]) # put in buffer and post increment 43 | add(r4, 4) 44 | cmp(r4, r5) # Check for buffer end 45 | ble(NOLOOP) 46 | mov(r4, r3) # Incremented past end: point to start 47 | label(NOLOOP) 48 | str(r4, [r0, 8]) # Save the insertion point for next call 49 | ldr(r1, [r0, 0]) # Element count 50 | sub(r1, 3) # No. of data points 51 | mov(r0, r7) # The sum 52 | sdiv(r0, r0, r1) # r0 = r0//r1 53 | -------------------------------------------------------------------------------- /avg_pico.py: -------------------------------------------------------------------------------- 1 | # Implementation of moving average filter in Arm Thumb V6 assembler for Pico 2 | # Released under the MIT License (MIT). See LICENSE. 3 | # Copyright (c) 2021 Peter Hinch 4 | 5 | # Function arguments: 6 | # r0 is an integer scratchpad array. Must be of length 3 greater than 7 | # the number of values to be averaged. 8 | # On entry array[0] must hold the array length, other elements must be zero 9 | # r1 holds new data value 10 | 11 | # Return value: the current moving average 12 | 13 | # array[0] is array length, array[1] is the current sum, array[2] the insertion point 14 | # r2 holds the length of the coefficient array 15 | # Pointers (byte addresses) 16 | # r3 start of ring buffer 17 | # r4 insertion point (post increment) 18 | # r5 last location of ring buffer 19 | # Other registers 20 | # r7 temporary store for result 21 | 22 | @micropython.asm_thumb 23 | def avg(r0, r1, r2): 24 | push({r2}) 25 | mov(r3, r0) 26 | add(r3, 12) # r3 points to ring buffer start 27 | ldr(r2, [r0, 0]) # Element count 28 | sub(r2, 4) # Offset in words to buffer end 29 | add(r2, r2, r2) 30 | add(r2, r2, r2) # convert to bytes 31 | add(r5, r2, r3) # r5 points to ring buffer end (last valid address) 32 | ldr(r4, [r0, 8]) # Current insertion point address 33 | cmp(r4, 0) # If it's zero we need to initialise 34 | bne(INIT) 35 | mov(r4, r3) # Initialise: point to buffer start 36 | label(INIT) 37 | ldr(r7, [r0, 4]) # get current sum 38 | ldr(r6, [r4, 0]) 39 | sub(r7, r7, r6) # deduct oldest value 40 | add(r7, r7, r1) # add newest value 41 | str(r7, [r0, 4]) # put sum back 42 | str(r1, [r4, 0]) # put in buffer and post increment 43 | add(r4, 4) 44 | cmp(r4, r5) # Check for buffer end 45 | ble(NOLOOP) 46 | mov(r4, r3) # Incremented past end: point to start 47 | label(NOLOOP) 48 | str(r4, [r0, 8]) # Save the insertion point for next call 49 | ldr(r1, [r0, 0]) # Element count 50 | sub(r1, 3) # No. of data points 51 | mov(r0, r7) # The sum 52 | pop({r2}) 53 | asr(r0, r2) # Scale (r0 = r0//r1 Unsupported by arm V6) 54 | -------------------------------------------------------------------------------- /avgtest.py: -------------------------------------------------------------------------------- 1 | # Demo program for moving average filter 2 | # Author: Peter Hinch 3 | # Released under the MIT License (MIT). See LICENSE. 4 | # Copyright (c) 2021 Peter Hinch 5 | # 16th Dec 2021 6 | 7 | import array 8 | from time import ticks_us, ticks_diff 9 | from avg import avg 10 | 11 | data = array.array('i', [0]*13) # Average over ten samples 12 | data[0] = len(data) 13 | 14 | def test(): 15 | for x in range(12): 16 | print(avg(data, 1000)) 17 | for x in range(12): 18 | print(avg(data, 0)) 19 | 20 | def timing(): 21 | t = ticks_us() 22 | avg(data, 10) 23 | t1 = ticks_diff(ticks_us(), t) # Time for one call with timing overheads 24 | t = ticks_us() 25 | avg(data, 10) 26 | avg(data, 10) 27 | t2 = ticks_diff(ticks_us(), t) # Time for two calls with timing overheads 28 | print(t2-t1,"uS") # Time to execute the avg() call 29 | 30 | test() 31 | print("Timing test") 32 | timing() 33 | 34 | -------------------------------------------------------------------------------- /avgtest_pico.py: -------------------------------------------------------------------------------- 1 | # Demo program for moving average filter 2 | # Author: Peter Hinch 3 | # Released under the MIT License (MIT). See LICENSE. 4 | # Copyright (c) 2021 Peter Hinch 5 | # 16th Dec 2021 6 | 7 | import array 8 | from time import ticks_us, ticks_diff 9 | from avg_pico import avg 10 | 11 | data = array.array('i', (0 for _ in range(19))) # Average over 16 samples 12 | data[0] = len(data) 13 | 14 | def test(): 15 | for x in range(16): 16 | print(avg(data, 1000, 4)) # Scale by 4 bits (divide by 16) 17 | for x in range(18): 18 | print(avg(data, 0, 4)) 19 | 20 | def timing(): 21 | t = ticks_us() 22 | avg(data, 10, 4) 23 | t1 = ticks_diff(ticks_us(), t) # Time for one call with timing overheads 24 | t = ticks_us() 25 | avg(data, 10, 4) 26 | avg(data, 10, 4) 27 | t2 = ticks_diff(ticks_us(), t) # Time for two calls with timing overheads 28 | print(t2-t1,"uS") # Time to execute the avg() call 29 | 30 | test() 31 | print("Timing test") 32 | timing() 33 | 34 | -------------------------------------------------------------------------------- /coeff_format.py: -------------------------------------------------------------------------------- 1 | # Utility to convert a set of FIR coefficients into Python code. 2 | # Cut and paste the (16 bit integer) coeffs from 3 | # http://t-filter.appspot.com/fir/index.html into a file. This will 4 | # have one coefficient per line. 5 | # This will create a readable output Python file defining the array. 6 | 7 | # Author: Peter Hinch 10th Feb 2015 8 | import sys 9 | 10 | def r(infile, outfile): 11 | with open(infile, "r") as f: 12 | with open(outfile, "w") as g: 13 | st = "import array\ncoeffs = array.array('i', (" 14 | g.write(st) 15 | x = 0 16 | st = "" 17 | while st == "": 18 | st = f.readline().strip() # ignore leading whitespace 19 | starting = True 20 | while st != "": 21 | if starting: 22 | starting = False 23 | else: 24 | g.write(",") 25 | x += 1 26 | g.write(st) 27 | if x % 15 == 0: 28 | g.write("\n ") 29 | st = f.readline().strip() 30 | g.write("))\n\n") 31 | 32 | def main(): 33 | if len(sys.argv) != 3 or sys.argv[0] == "--help": 34 | print("Usage: python3 coeff_format.py data_filename python_filename") 35 | else: 36 | r(sys.argv[1], sys.argv[2]) 37 | 38 | main() 39 | 40 | -------------------------------------------------------------------------------- /fir.py: -------------------------------------------------------------------------------- 1 | # Implementation of FIR filter in Arm Thumb assembler 2 | # Released under the MIT License (MIT). See LICENSE. 3 | # Copyright (c) 2021 Peter Hinch 4 | # 22nd Dec 2021: update to support ARMV6. 5 | # Calculate coefficients here: http://t-filter.appspot.com/fir/index.html 6 | 7 | # Function arguments: 8 | # r1 is an integer array of coefficients 9 | # r0 is an integer scratchpad array. Must be of length 3 greater than the # of coefficients. 10 | # Entry: 11 | # array[0] must hold the array length 12 | # array[1] the number of bits to scale (right shift) the result (0-31). 13 | # other elements must be zero 14 | 15 | # Run conditions 16 | # array[2] holds the insertion point address 17 | # r2 holds the new data value 18 | # Register usage (Buffer) 19 | # r0 Scratchpad 20 | # r2 new data value 21 | # r3 ring buffer start 22 | # r4 insertion point (post increment) 23 | # r5 last location in ring buffer 24 | 25 | # Register usage (filter) 26 | # r0 accumulator for result 27 | # r1 coefficient array 28 | # r2 current coeff 29 | # r3 ring buffer start 30 | # r4 insertion point (post increment) 31 | # r5 last location in ring buffer 32 | # r6 data point counter 33 | # r7 curent data value 34 | # r8 scaling value 35 | 36 | @micropython.asm_thumb 37 | def fir(r0, r1, r2): 38 | mov(r3, r8) # For Pico: can't push({r8}). r0-r7 only. 39 | push({r3}) 40 | ldr(r7, [r0, 0]) # Array length 41 | mov(r6, r7) # Copy for filter 42 | mov(r3, r0) 43 | add(r3, 12) # r3 points to ring buffer start 44 | sub(r7, 1) 45 | add(r7, r7, r7) 46 | add(r7, r7, r7) # convert to bytes 47 | add(r5, r7, r3) # r5 points to ring buffer end (last valid address) 48 | ldr(r4, [r0, 8]) # Current insertion point address 49 | cmp(r4, 0) # If it's zero we need to initialise 50 | bne(INITIALISED) 51 | mov(r4, r3) # Initialise: point to buffer start 52 | label(INITIALISED) 53 | str(r2, [r4, 0]) # put new data in buffer and post increment 54 | add(r4, 4) 55 | cmp(r4, r5) # Check for buffer end 56 | ble(BUFOK) 57 | mov(r4, r3) # Incremented past end: point to start 58 | label(BUFOK) 59 | str(r4, [r0, 8]) # Save the insertion point for next call 60 | # *** Filter *** 61 | ldr(r0, [r0, 4]) # Bits to shift 62 | mov(r8, r0) 63 | mov(r0, 0) # r0 Accumulator 64 | label(FILT) 65 | ldr(r7, [r4, 0]) # r7 Data point (start with oldest) 66 | add(r4, 4) 67 | cmp(r4, r5) 68 | ble(NOLOOP1) 69 | mov(r4, r3) 70 | label(NOLOOP1) 71 | ldr(r2, [r1, 0]) # r2 Coefficient 72 | add(r1, 4) # Point to next coeff 73 | mul(r2, r7) 74 | mov(r7, r8) 75 | asr(r2, r7) # Scale result before summing 76 | add(r0, r2, r0) 77 | sub(r6, 1) 78 | bne(FILT) # > 0. bpl branched when >= 0 79 | pop({r3}) 80 | mov(r8, r3) # Restore R8 81 | 82 | -------------------------------------------------------------------------------- /fir_py.py: -------------------------------------------------------------------------------- 1 | # fir_py.py FIR filter implemented with Viper 2 | # Released under the MIT License (MIT). See LICENSE. 3 | # Copyright (c) 2021 Peter Hinch 4 | 5 | from array import array 6 | # Closures under Viper, see 7 | # https://github.com/micropython/micropython/issues/8086 8 | # The workround seems fragile so I'm using an array to hold state 9 | def create_fir(coeffs, shift): 10 | nc = len(coeffs) 11 | data = array('i', (0 for _ in range(nc))) 12 | ctrl = array('i', (0, shift, nc)) 13 | @micropython.viper 14 | def inner(val : int) -> int: 15 | buf = ptr32(data) 16 | ctl = ptr32(ctrl) 17 | co = ptr32(coeffs) 18 | shift : int = ctl[1] 19 | nc : int = ctl[2] 20 | end : int = nc - 1 21 | i : int = ctl[0] 22 | buf[i] = val 23 | i = (i + 1) if (i < end) else 0 24 | ctl[0] = i 25 | res : int = 0 26 | for x in range(nc): 27 | res += (co[x] * buf[i]) >> shift 28 | i = (i + 1) if (i < end) else 0 29 | return res 30 | return inner 31 | -------------------------------------------------------------------------------- /firtest.py: -------------------------------------------------------------------------------- 1 | # Test functions for FIR filter 2 | # Released under the MIT License (MIT). See LICENSE. 3 | # Copyright (c) 2021 Peter Hinch 4 | 5 | import array 6 | from time import ticks_us, ticks_diff 7 | from fir import fir 8 | 9 | # Coefficient options 10 | # 41 tap low pass filter, 2dB ripple 60dB stop 11 | c = [-176, -723, -1184, -868, 244, 910, 165, -1013, -693, 12 | 977, 1398, -615, -2211, -257, 3028, 1952, -3729, -5500, 13 | 4201, 20355, 28401, 20355, 4201, -5500, -3729, 1952, 14 | 3028, -257, -2211, -615, 1398, 977, -693, -1013, 165, 15 | 910, 244, -868, -1184, -723, -176] 16 | 17 | # 21 tap LPF 18 | d = [-1318, -3829, -4009, -717, 3359, 2177, -3706, -5613, 4154, 20372, 28471, 20372, 4154, -5613, -3706, 19 | 2177, 3359, -717, -4009, -3829, -1318] 20 | # 109 tap LPF 21 | e = [-24, 89, 69, 78, 95, 115, 135, 154, 171, 185, 196, 202, 201, 22 | 194, 178, 155, 122, 81, 31, -26, -91, -160, -232, -306, -378, 23 | -446, -506, -555, -591, -610, -608, -584, -535, -460, -357, 24 | -225, -66, 121, 333, 568, 823, 1094, 1375, 1663, 1952, 2237, 25 | 2510, 2768, 3004, 3213, 3391, 3534, 3638, 3702, 3723, 3702, 26 | 3638, 3534, 3391, 3213, 3004, 2768, 2510, 2237, 1952, 1663, 27 | 1375, 1094, 823, 568, 333, 121, -66, -225, -357, -460, -535, 28 | -584, -608, -610, -591, -555, -506, -446, -378, -306, -232, 29 | -160, -91, -26, 31, 81, 122, 155, 178, 194, 201, 202, 196, 30 | 185, 171, 154, 135, 115, 95, 78, 69, 89, -24] 31 | 32 | # Initialisation 33 | coeffs = array.array('i', d) 34 | ncoeffs = len(coeffs) 35 | data = array.array('i', [0]*(ncoeffs +3)) 36 | data[0] = ncoeffs 37 | data[1] = 1 # Try a single bit shift 38 | 39 | def test(): # Impulse response replays coeffs*impulse_size >> scale 40 | print(fir(data, coeffs, 100)) 41 | for n in range(len(coeffs)+3): 42 | print(fir(data, coeffs, 0)) 43 | 44 | def timing(): # Test removes overhead of pyb function calls 45 | t = ticks_us() 46 | fir(data, coeffs, 100) 47 | t1 = ticks_diff(ticks_us(), t) 48 | t = ticks_us() 49 | fir(data, coeffs, 100) 50 | fir(data, coeffs, 100) 51 | t2 = ticks_diff(ticks_us(), t) 52 | print(t2-t1,"uS") 53 | # Results: 14uS for a 21 tap filter, 16uS for 41 taps, 23uS for 109 (!) 54 | # Time = 12 + 0.102N uS where N = no. of taps 55 | # Tests on large numbers suggest the formula t = 10 + 0.12N is closer 56 | test() 57 | print("Done! Timing:") 58 | timing() 59 | 60 | -------------------------------------------------------------------------------- /images/lpf_bode.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/peterhinch/micropython-filters/4e0b737574073bab36ec1c776e8dfb80b8fe5f9f/images/lpf_bode.jpg -------------------------------------------------------------------------------- /images/lpf_nyquist.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/peterhinch/micropython-filters/4e0b737574073bab36ec1c776e8dfb80b8fe5f9f/images/lpf_nyquist.jpg -------------------------------------------------------------------------------- /lpf.py: -------------------------------------------------------------------------------- 1 | # Demo program for FIR filter module 2 | # Author: Peter Hinch 3 | # 12th Feb 2015 4 | # Outputs a swept frequency sine wave on Dac1 5 | # Timer interrupt reads the analog input, filters it, and outputs the result on Dac 2. 6 | # Requires a link between X5 and X7 7 | 8 | import math 9 | import pyb 10 | import array 11 | from fir import fir 12 | import micropython 13 | micropython.alloc_emergency_exception_buf(100) 14 | 15 | # Define hardware 16 | dac1 = pyb.DAC(1) 17 | dac2 = pyb.DAC(2) 18 | adc = pyb.ADC("X7") 19 | tim = pyb.Timer(4, freq=2000) # Sampling freq 10KHz is about the limit 14KHz without constraint 20 | 21 | # Data for FIR filter Pass (@2Ksps) 0-40Hz Stop 80Hz-> 22 | coeffs = array.array('i', (72, 47, 61, 75, 90, 105, 119, 132, 142, 149, 152, 149, 23 | 140, 125, 102, 71, 33, -12, -65, -123, -187, -254, -322, -389, -453, -511, -561, 24 | -599, -622, -628, -615, -579, -519, -435, -324, -187, -23, 165, 375, 607, 855, 25 | 1118, 1389, 1666, 1941, 2212, 2472, 2715, 2938, 3135, 3303, 3437, 3535, 3594, 26 | 3614, 3594, 3535, 3437, 3303, 3135, 2938, 2715, 2472, 2212, 1941, 1666, 1389, 27 | 1118, 855, 607, 375, 165, -23, -187, -324, -435, -519, -579, -615, -628, -622, 28 | -599, -561, -511, -453, -389, -322, -254, -187, -123, -65, -12, 33, 71, 102, 125, 29 | 140, 149, 152, 149, 142, 132, 119, 105, 90, 75, 61, 47, 72)) 30 | ncoeffs = len(coeffs) 31 | data = array.array('i', [0]*(ncoeffs +3)) # Scratchpad must be three larger than coeffs 32 | data[0] = ncoeffs 33 | data[1] = 16 34 | 35 | # Data input, filter and output 36 | # The constraint is a convenience to ensure that any inadvertent overflow shows as 37 | # clipping on the 'scope. If you get the scaling right you can eliminate it... 38 | def cb(timer): 39 | val = fir(data, coeffs, adc.read()) // 16 # Filter amd scale 40 | dac2.write(max(0, min(255, val))) # Constrain, shift (no DC from bandpass) and output 41 | 42 | tim.callback(cb) 43 | 44 | # Sweep generator 45 | def sine_sweep(start, end, mult): # Emit sinewave on DAC1 46 | buf = bytearray(100) 47 | for i in range(len(buf)): 48 | buf[i] = 128 + int(110 * math.sin(2 * math.pi * i / len(buf))) 49 | 50 | freq = start 51 | while True: 52 | dac1.write_timed(buf, int(freq) * len(buf), mode=pyb.DAC.CIRCULAR) 53 | print(freq, "Hz") 54 | pyb.delay(2500) 55 | freq *= mult 56 | if freq > end: 57 | freq = start 58 | 59 | sine_sweep(10, 400, 1.33) 60 | 61 | -------------------------------------------------------------------------------- /non_realtime/FILT.md: -------------------------------------------------------------------------------- 1 | # 1. Fast filtering of sample arrays 2 | 3 | A function `dcf` is provided which performs discrete convolution on an array of 4 | integer samples acquired from an ADC. This enables the implementation of finite 5 | impulse response (FIR) filters with characteristics including high pass, low 6 | pass, band pass and band stop. It achieves millisecond and sub-ms level 7 | performance on platforms using STM chips (for example the Pyboard). To achieve 8 | this it uses the MicroPython inline assembler for the ARM Thumb instruction 9 | set. It uses the hardware floating point unit. 10 | 11 | It can also be used for techniques such as correlation to detect signals in the 12 | presence of noise. 13 | 14 | A version `dcf_fp` is effectively identical but accepts an array of floats as 15 | input. 16 | 17 | Section 2 describes the underlying principles with section 3 providing the 18 | details of the function's usage. 19 | 20 | ## 1.1 Files 21 | 22 | * `filt.py` Module providing the `dcf` function and the `dcf_fp` float version. 23 | * `filt_test.py` Test/demo of FIR filtering. 24 | * `coeffs.py` Coefficients for the above. 25 | * `correlate.py` Test/demo of correlation See [section 3.2](./FILT.md#32-correlation-test). 26 | * `autocorrelate.py` Demo of autocorrelation using `dcf_fp` [section 3.3](./FILT.md#33-autocorrelation). 27 | * `samples.py` Example of the output of `autocorrelate.py`. 28 | * `filt_test_all` Test suite for `dcf` and `dcf_fp` functions. 29 | * `correlate.jpg` Image showing data recovery from noise. 30 | 31 | The test programs use simulated data and run on import. See code comments for 32 | documentation. 33 | 34 | # 2. Filtering Sample Arrays 35 | 36 | One application of the `dcf` function is to perform FIR filtering on an array 37 | of input samples acquired from an ADC. An array of coefficients determines the 38 | filter characteristics. If there are `N` samples and `P` coefficients, the most 39 | recent `P` samples are each multiplied by its corresponding coefficient. The 40 | summation of the results becomes the most recent element in the output array. 41 | This process is repeated for successively earlier initial samples. See 42 | [Wikipedia](https://en.wikipedia.org/wiki/Finite_impulse_response). 43 | 44 | Mathematically this process is similar to convolution, cross-correlation and 45 | auto-correlation and the function may be employed for those purposes. In 46 | particular correlation may be used to detect an expected signal in the presence 47 | of noise. 48 | 49 | ## 2.1 Aliasing and Decimation 50 | 51 | In any sampled data system the maximum frequency which may be accurately 52 | represented is given by `Fmax = Fs/2` where `Fs` is the sampling rate. `Fmax` 53 | is known as the Nyquist rate. If analog signals with a higher frequency are 54 | present at the ADC input, these will be 'aliased', appearing as signals below 55 | the Nyquist rate. A low pass analog filter should be used prior to the ADC to 56 | limit the amplitide of aliased signals. 57 | 58 | In some applications a signal is initially sampled at a high rate before being 59 | filtered and down-sampled to the intended rate. Such a filter is known as a 60 | decimation filter. The anti-aliasing filter comprises a relatively simple 61 | analog filter with further low pass filtering being performed in the digital 62 | domain. The `dcf` function provides optional decimation. See 63 | [Wikipedia](https://en.wikipedia.org/wiki/Decimation_(signal_processing)). 64 | 65 | ## 2.2 Bias removal 66 | 67 | Typically the analog signal has a DC bias to ensure that all samples lie within 68 | range of the ADC. In most applications the DC bias should be removed prior 69 | to processing so that signals are symmetrical about 0.0. Usually this is done 70 | by subracting the calculated value of the DC bias (e.g. 2048 for a 12 bit ADC 71 | biassed at its mid point). In cases where the DC bias is not known at design 72 | time it may be derived from the mean of the sample set. The function provides 73 | for both cases. 74 | 75 | The function also provides for cases where bias removal is inappropriate - in 76 | particular filters whose aim is to perform a measurement of the DC level in the 77 | presence of noise. 78 | 79 | ## 2.3 Circular or Linear Convolution 80 | 81 | The analog signal being sampled can be considered to be continuous or 82 | discontinuous. An example of a continuous signal is where the waveform being 83 | studied has a known frequency, and the sample rate has been chosen to sample an 84 | integer number of cycles. Another example is a study of acoustic background 85 | noise: though the signal isn't formally repetitive it may be valid to process 86 | it as if it were. In these cases `dcf` can be configured to treat the input 87 | buffer as a circular buffer; an array of `N` samples will produce `N` results. 88 | 89 | A sample array considered to be non-repetitive will not process the oldest 90 | samples: an array of `N` samples processed with `P` coefficients will produce 91 | `N-P` results. In the non-repetitive case processing of the oldest samples 92 | cannot be performed: this occurs once the number of outstanding samples is less 93 | than the number of coefficients. 94 | 95 | ## 2.4 Design of FIR filters 96 | 97 | FIR filters can be designed for low pass, high pass, bandpass or band stop 98 | applications and the site [TFilter](http://t-filter.engineerjs.com/) enables 99 | the coefficients to be computed. For `dcf` select the `double` option to 100 | make it produce floating point results. The C/C++ option is easy to convert to 101 | Python. 102 | 103 | # 3. Function dcf (and dcf_fp) 104 | 105 | `dcf` performs an FIR filtering operation (discrete convolution) on an integer 106 | (half word) array of input samples and an array of floating point coefficients. 107 | Results are placed in an array of floats, with an option to copy back to the 108 | source array. Arrays are ordered by time with the earliest entry in element 0. 109 | 110 | Samples must be >= 0, as received from an ADC. The alternative implementation 111 | `dcf_fp` in the file `filt_fp` is identical except that the input array is an 112 | array of floats with no value restriction. 113 | 114 | If the number of samples is `N` and a decimation factor `D` is applied, the 115 | number of result elements is `N // D`. 116 | 117 | Results are "left justified" in the result array, with the oldest result in 118 | element 0. In cases where there are fewer result values than input samples, 119 | this allows the result array to be smaller than the sample array, saving RAM. 120 | 121 | The function does not allocate memory. 122 | 123 | Args: 124 | `r0` The input sample array. An array of unsigned half words (`dcf`) for 125 | `dcf_fp` an array of floats is required. 126 | `r1` The result array. An array of floats. In the normal case where the 127 | decimation factor is 1 this should be the same length as the sample array. If 128 | decimating by N its length must be >= (no. of samples)/N. 129 | `r2` Coefficients. An array of floats. 130 | `r3` Setup: an integer array of 5 elements. 131 | 132 | `Setup[0]` Length of the sample array. 133 | `Setup[1]` Length of the coefficient array. Must be <= length of sample array. 134 | `Setup[2]` Flags (bits controlling algorithm behaviour). See below. 135 | `Setup[3]` Decimation factor. For no decimation set to 1. 136 | `Setup[4]` Offset. DC offset of input samples. Typically 2048. If -1 the 137 | average value of the samples is calculated and assumed to be the offset. 138 | 139 | Return value: 140 | `n_results` The number of result values. This depends on `wrap` and the 141 | decimation value. Valid data occupy result array elements 0 to `n_results -1`. 142 | Any subsequent output array elements will retain their original contents. 143 | 144 | Flags: 145 | `b0` Wrap. Determines circular or linear processing. See section 3.1.1. 146 | `b1` Scale. If set, scale all results by a factor. If used, the (float) scale 147 | factor must be placed in element 0 of the result array. All results are 148 | multiplied by this factor. Scaling is a convenience feature: the same result 149 | may be achieved by muliplying all the coefficients by the factor. 150 | `b2` Reverse. Determines whether coefficients are applied in time order or 151 | reverse time order. If the coefficients are viewed as a time sequence (as in 152 | correlation or convolution), by default they are applied with the most recent 153 | sample being matched with the most recent coefficient. So correlation is 154 | performed in the default (forward) order. If using coefficients from 155 | [TFilter](http://t-filter.engineerjs.com/) for FIR filtering the order is 156 | immaterial as these arrays are symmetrical (linear phase filters). For 157 | convolution the reverse order should be selected. 158 | `b3` Copy. Determines whether results are copied back to the sample array. If 159 | this is selected `dcf` converts the results to integers. In either case (`dcf` 160 | or `dcf_fp`) the DC bias is restored. If decimation is selected elements 0 to 161 | `N // D` will be populated. Remaining samples will be set to the mean (DC bias) 162 | level. 163 | 164 | Constants `WRAP`, `SCALE`, `REVERSE` and `COPY` are provided enabling the flags 165 | to be set by the logical `or` of the chosen constants. Typical setup code is as 166 | below (`bufin` is the sample array, `coeffs` the coefficients and `op` is the 167 | output array): 168 | 169 | ```python 170 | setup = array('i', (0 for _ in range(5))) 171 | setup[0] = len(bufin) # Number of samples 172 | setup[1] = len(coeffs) # Number of coefficients 173 | setup[2] = SCALE | WRAP | COPY 174 | setup[3] = 1 # Decimation factor 175 | setup[4] = -1 # Calculate the offset from the mean 176 | op[0] = 1.1737 # Scaling factor in result array[0] 177 | ``` 178 | 179 | The result array is in time order with the oldest result in `result[0]`. If the 180 | result array is longer than required, elements after the newest result will not 181 | be altered. 182 | 183 | These assembler functions are built for speed, not for comfort. There are 184 | minimal checks on passed arguments. Required checking should be done in Python; 185 | errors (such as output array too small) will result in a crash. Supplying `dcf` 186 | with samples < 0 won't crash but will produce garbage. 187 | 188 | ## 3.1 Algorithm 189 | 190 | For users unfamiliar with the maths the basic algorithm is illustrated by this 191 | Python code. The input, output and coefficient arrays are args `ip`, `op`, 192 | `coeffs`. Circular processing is controlled by `wrap` and `dc` contains the DC 193 | bias (2048 for a 12 bit ADC). This illustration is much simplified and does not 194 | include decimation. Coefficient processing is only in normal order; it does not 195 | left justify the results. 196 | 197 | ```python 198 | def filt(ip, op, coeffs, scale, wrap, dc=2048): 199 | iplen = len(ip) # array of input samples 200 | last = -1 if wrap else len(coeffs) - 2 201 | for idx in range(iplen -1, last, -1): # most recent first 202 | res = 0.0 203 | cidx = idx 204 | for x in range(len(coeffs) -1, -1, -1): # Array of coeffs (float) 205 | res += (ip[cidx] - dc) * coeff[idx] # end of array first 206 | cidx -= 1 207 | cidx %= iplen # Circular processing 208 | op[idx] = res * scale # Float o/p array 209 | for idx, entry in enumerate(op): # Copy back to i/p array, restore DC 210 | ip[idx] = int(entry) + dc 211 | ``` 212 | 213 | The first sample in the sample array is presumed to be the oldest. The filter 214 | algorithm starts with the newest sample with the result going into the last 215 | element of the result array: time ordering is preserved. If there are `N` 216 | coefficients, the current sample and `N-1` predecessors are each multiplied by 217 | the matching coefficient, with the result being the sum of these products. This 218 | result is multiplied by the scaling factor and placed in the result array, in 219 | the position matching the original sample. 220 | 221 | In the above example and in `dcf` (in normal order) coefficients are applied in 222 | time order. Thus if the coefficient array contains `n` elements numbered from 0 223 | to n-1, when processing `sample[x]`, `result[x]` is 224 | `sample[x]*coeff[n -1] + sample[x-1]*coeff[n-2] + sample[x-2]*coeff[n-3]...` 225 | 226 | In the Python code DC bias is subtracted from each sample before processing - 227 | `dcf` uses `Setup[4]` to control this behaviour. 228 | 229 | Results in the output array do not have DC bias restored - they are referenced 230 | to zero. 231 | 232 | ### 3.1.1 Wrap 233 | 234 | Where `wrap` is `True` the outcome is an output array where each element 235 | corresponds to an element in the input array. 236 | 237 | Assume there are `P` coefficients. If `wrap` is `False`, as the algorithm steps 238 | back to earlier samples, a point arises when there are fewer than `P` samples. 239 | In this case it is not possible to calculate a vaild result. `N` samples 240 | processed with `P` coefficients will produce `N-P+1` results. Element 0 of the 241 | result array will hold the oldest valid result. Elements after the newest 242 | result will be unchanged. 243 | 244 | See [section 2.3](./FILT.md#23-circular-or-linear-convolution) for a discussion 245 | of this from an application perspective. 246 | 247 | ### 3.1.2 Decimation 248 | 249 | In normal processing `Setup[3]` should be 1. If it is set to a value `D` where 250 | `D > 1` only `N//D` output samples will be produced. These will be placed in 251 | elements 0 upwards of the output array. Subsequent elements of the result array 252 | will be unchanged. 253 | 254 | Decimation saves processing time (fewer results are computed) and RAM (a 255 | smaller output array may be declared). 256 | 257 | ### 3.1.3 Copy 258 | 259 | If `Setup[2]` has the `copy` bit set results are copied back to the original 260 | sample array: to do this the DC bias is restored and the result converted to a 261 | 16-bit integer. The bias value used is that employed in the filter computation: 262 | either a specified value or a measured mean. 263 | 264 | If decimation is selected such that `Q` results are computed, the first `Q` 265 | entries in the sample array will contain valid data. Subsequent elements will 266 | be set to the DC bias value. 267 | 268 | See [section 2.1 Aliasing and decimation](./FILT.md#21-aliasing-and-decimation) 269 | for a discussion of this. 270 | 271 | ### 3.1.3 Performance 272 | 273 | On a Pyboard V1.1 a convolution of 128 samples with 41 coefficients, computing 274 | the mean, using wrap and copying all samples back, took 912μs. This compares 275 | with 128ms for a MicroPython version using the standard code emitter. While 276 | the time is dependent on the number of samples and coefficients it is otherwise 277 | something of a worst case. It will be reduced if a fixed offset, no wrap or 278 | decimation is used. 279 | 280 | A correlation of 1000 samples with a 50 coefficient signal takes 7.6ms. 281 | 282 | It is worth noting that there are faster convolution algorithms. The "brute 283 | force and ignorance" approach used here has the merit of being simple to code 284 | in Assembler with the drawback of O(N^2) performance. Algorithms with 285 | O(N.log N) performance exist, at some cost in code complexity. Typical 286 | microcontroller applications may have anything up to a few thousand samples but 287 | few coefficients. This contrasts with statistical applications where both 288 | sample sets may be very large. 289 | 290 | ## 3.2 Correlation test 291 | 292 | The program `correlate.py` uses a signal designed to have a zero DC component, 293 | a runlength limited to two consecutive 1's or 0's, and an autocorrelation 294 | function approximating to an impulse function. It adds it in to a long sample 295 | of analog noise and then uses correlation to determine its location in the 296 | summed signal. 297 | 298 | The graph below shows a 100 sample snapshot of the simulated datastream which 299 | includes the target signal. This can reliably be identified by `correlate.py` 300 | with precise timing. In the graph below the blue line represents the signal and 301 | the red the correlation. The highest peak occurs at the correct sample at the 302 | point in time when the entire expected signal has been received. 303 | 304 | ![graph](./correlate.jpg) 305 | 306 | As a simulation `correlate.py` is probably unrealistic from an engineering 307 | perspective as sampling is at exactly the Nyquist rate. I have not yet tested 308 | correlation with real hardware but I can envisage issues if the timing of the 309 | received signal is such that state changes in the expected signal happen to 310 | occur close to the sample time. To avoid uncertainty caused by signal timing 311 | jitter it is likely that sampling should be done at N*Nyquist. Two solutions 312 | to processing the signal suggest themselves. 313 | 314 | The discussion below assumes 2*Nyquist. 315 | 316 | One approach is to double the length of the coefficient (expected signal) array 317 | repeating each entry so that the time sequence in the coefficient array matches 318 | that in the sample array. Potentially the correlation maximum will occur on two 319 | successive samples but in terms of time (because of the sample rate) this does 320 | not imply a loss of precision. 321 | 322 | The other is to first run a decimation filter to reduce the rate to Nyquist. A 323 | decimation filter will introduce a time delay, but with a linear phase filter 324 | as produced by TFilter this is fixed and frequency-independent. Decimation is 325 | likely to produce the best result in the presence of noise because it is a 326 | filter: a suitable low pass design will remove in-band noise and aliased out of 327 | band frequencies. 328 | 329 | In terms of performance, if sampling at Nyquist takes time T the first approach 330 | will take 2T. The second will take T + Tf where Tf is the time to do the FIR 331 | filtering. Tf will be less than T if the number of filter coefficients is less 332 | than the number of samples in the expected signal. 333 | 334 | If sampling at N*Nyquist where N > 2 decimation is likely to win out. Even if 335 | the number of filter coefficients increases, a decimation filter is fast 336 | because it only computes a subset of the results. 337 | 338 | If anyone tries this before I do, please raise an issue describing your 339 | approach and I will amend this doc. 340 | 341 | ## 3.3 Autocorrelation 342 | 343 | The file `autocorrelate.py` demonstrates autocorrelation using `dcf_fp`, 344 | producing bit sequences with the following characteristics: 345 | 1. The sequence length is specified by the caller. 346 | 2. The number of 1's is equal to the number of 0's (no DC component). 347 | 3. The maximum run length (number of consecutive identical bits) is defined by 348 | the caller. 349 | 4. The autocorrelation function is progressively optimised. 350 | 351 | The aim is to produce a bitstream suitable for applying to physical transducers 352 | (or to radio links), such that the bitstream may be detected in the presence of 353 | noise. The ideal autocorrelation would be a pulse corresponding to a single 354 | sample, with 0 elsewhere. This cannot occur in practice but the algorithm 355 | produces a "best possible" solution quickly. 356 | 357 | The algorithm is stochastic. It creates a random bit pattern meeting the 358 | runlength criterion, discarding any with a DC component. It then performs a 359 | linear autocorrelation on it. The detection ratio (ratio of "best" result to 360 | the next largest result) is calculated. 361 | 362 | This is repeated until the specified runtime has elapsed, with successive bit 363 | patterns having the highest detection ratio achieved being written to the file. 364 | 365 | It produces a Python file on the Pyboard (default name `/sd/samples.py`). An 366 | example is provided to illustrate the format. 367 | -------------------------------------------------------------------------------- /non_realtime/autocorrelate.py: -------------------------------------------------------------------------------- 1 | # autocorrelate.py 2 | 3 | # Released under the MIT licence. 4 | # Copyright Peter Hinch 2018 5 | 6 | # Generates Python code providing a binary signal having good autocorrelation 7 | # characteristics. These comprise: 8 | # 1. An autocorrelation function approximating an impulse function. 9 | # 2. A limited maximum runlength (i.e. limited bandwidth). 10 | # 3. Zero DC component (no. of 1's == no. of zeros) 11 | # Note that specifying a large maximum runlength will probably result in an 12 | # actual runlength which is much shorter: the process is stochastic and long 13 | # runlengths are rare. 14 | 15 | # The Python code produced instantiates the following variables corresponding 16 | # to the best result achieved. 17 | # data a list of data values: these are 1 or -1. 18 | # detection_ratio: (value at actual match)/(next highest value) 19 | # runlength The actual maximum runlength (<= the specified RL). 20 | 21 | # A random sequence of N values constrained to +1 or -1 is generated which meets 22 | # the RL and zero DC requirements. 23 | # The coefficients are set to that sequence (autocorrelation). 24 | # The input data sequence is N zeros followed by the sequence. 25 | # Linear cross correlation is performed and the performance checked. 26 | # The current best sequence is written to file. 27 | 28 | import utime 29 | import pyb 30 | from array import array 31 | from filt import dcf_fp, WRAP, SCALE, REVERSE 32 | 33 | def make_sample(bufin, coeffs, siglen, max_rl): 34 | actual_max_rl = 0 35 | while True: 36 | rl = 0 37 | for x in range(siglen): 38 | bit = 1 if pyb.rng() & 1 else -1 39 | if rl < max_rl: 40 | coeffs[x] = bit 41 | last_bit = bit 42 | if rl > actual_max_rl: 43 | actual_max_rl = rl 44 | rl += 1 45 | else: 46 | last_bit *= -1 47 | coeffs[x] = last_bit 48 | rl = 1 49 | 50 | if sum(coeffs) == 0: # No DC component 51 | # Signal is N zeros followed by N values of +-1 52 | # For autocorrelation signal == coeffs 53 | for x in range(siglen): 54 | bufin[x] = 0 55 | for x in range(siglen): 56 | bufin[x + siglen] = coeffs[x] # range +-1 57 | return actual_max_rl 58 | 59 | def main(filename='/sd/samples.py', runtime=60, siglen=50, max_rl=6): 60 | if siglen % 2 == 1: 61 | print('Signal length must be even to meet zero DC criterion.') 62 | return 63 | if max_rl < 2: 64 | print('Max runlength must be >= 2.') 65 | return 66 | # Setup for correlation 67 | setup = array('i', (0 for _ in range(5))) 68 | setup[0] = 2 * siglen # Input buffer length 69 | setup[1] = siglen # Coefficient buffer length 70 | setup[2] = 0 # Normal time order. No wrap. No copy back: I/P sample set unchanged 71 | setup[3] = 1 # No decimation. 72 | setup[4] = 0 # No offset. 73 | with open(filename, 'w') as f: 74 | f.write('# Code produced by autocorrelate.py\n') 75 | f.write('# Each list element comprises [[data...], detection_rato, runlength]\n') 76 | f.write('# where runlength is the actual maximum RL in the signal.\n') 77 | f.write('# Variables data, detection_ratio and runlength hold the values\n') 78 | f.write('# for the best detection_ratio achieved.\n') 79 | f.write('signal_length = {:d}\n'.format(siglen)) 80 | f.write('max_run_length = {:d}\n'.format(max_rl)) 81 | f.write('signals = [\n') 82 | bufin = array('f', (0 for i in range(2*siglen))) 83 | op = array('f', (0 for _ in range(2*siglen))) 84 | coeffs = array('f', (0 for _ in range(siglen))) 85 | 86 | start = utime.time() 87 | end = start + runtime 88 | count = 0 # Candidate count 89 | last_best = siglen # Count of mismatched bits in worst invalid detection 90 | det_ratio = 0 # Detection ratio: (value at actual match)/(next highest value) 91 | while utime.time() < end: 92 | # Populate bufin and coeffs with a signal 93 | # meeting runlength and zero DC criteria 94 | actual_rl = make_sample(bufin, coeffs, siglen, max_rl) 95 | count += 1 96 | n_results = dcf_fp(bufin, op, coeffs, setup) 97 | 98 | maxop = max(op) # Highest correlation 99 | nextop = 0 # next highest (i.e. worst invalid) correlation 100 | for x in range(n_results): 101 | res = op[x] 102 | if res < maxop and res > nextop: 103 | nextop = res 104 | if nextop < last_best: # new best candidate 105 | last_best = nextop 106 | for x in range(n_results): 107 | res = op[x] 108 | print('{:3d} {:8.1f}'.format(x, res)) 109 | s = 'Max correlation {:5.1f} Next largest {:5.1f} Detection ratio {:5.1f}.' 110 | print(s.format(maxop, nextop, maxop/nextop)) 111 | print('runtime (secs)', (utime.time() - start)) 112 | el_count = 0 113 | with open(filename, 'a') as f: 114 | f.write('[[') 115 | for x in coeffs: 116 | f.write(str(int(x))) # 1 or -1 117 | f.write(',') 118 | el_count += 1 119 | el_count %= 16 120 | if el_count == 0: 121 | f.write('\n') 122 | # Note that actual max runlength can be < specified value 123 | # so write out actual value. 124 | det_ratio = maxop/nextop if nextop != 0 else 0 # Pathological data ?? 125 | f.write('],{:5.1f}, {:d}],\n'.format(det_ratio, actual_rl)) 126 | 127 | with open(filename, 'a') as f: 128 | f.write(']\n') 129 | f.write('data, detection_ratio, runlength = signals[-1]\n') 130 | fstr = 'Best detection ratio achieved: {:5.1f} match {:5.1f} mismatch {:5.1f}' 131 | print(fstr.format(det_ratio, maxop, last_best)) 132 | print('rl == {:d} patterns tested {:4d}.'.format(max_rl, count)) 133 | 134 | # rl == 2 yielded det ratio 10 (50/10 == 5 bits match) in 71s of 15 min run 135 | # main(filename = '/sd/rl_2.py', runtime=900, max_rl = 2) 136 | # Let it rip with any runlength and analyse the results afterwards: 137 | # best ratio was 12.5 with rl == 6 138 | #main(filename = '/sd/rl_any.py', runtime=3600, max_rl = 25) 139 | # Find best rl <= 3 sequence yielded det ratio 10 140 | #main(filename = '/sd/rl_3.py', runtime=60, max_rl = 3) 141 | # Find best rl <= 6 sequence 142 | main() 143 | -------------------------------------------------------------------------------- /non_realtime/coeffs.py: -------------------------------------------------------------------------------- 1 | # Filter coeffs for network analyser. Frozen. 2 | from array import array 3 | # http://t-filter.engineerjs.com/ 4 | # t-filter parameters based on 2KHz sampling 5 | 6 | # FIR filter designed with 7 | # http://t-filter.engineerjs.com/ http://t-filter.appspot.com 8 | 9 | # sampling frequency: 2000 Hz 10 | 11 | # 0 Hz - 160 Hz 12 | # gain = 0 13 | # desired attenuation = -50 dB 14 | # actual attenuation = -58.773432260359314 dB 15 | 16 | # 245 Hz - 255 Hz 17 | # gain = 1 18 | # desired ripple = 3 dB 19 | # actual ripple = 0.4395212955083323 dB 20 | 21 | # 340 Hz - 1000 Hz 22 | # gain = 0 23 | # desired attenuation = -50 dB 24 | # actual attenuation = -58.773432260359314 dB 25 | 26 | coeffs_8a = array('f', ( 27 | -0.0008646866700221276, 28 | -0.00007789338800231479, 29 | 0.0014455418248231421, 30 | 0.00324942874567572, 31 | 0.0034335420218244736, 32 | 0.0003696228529220283, 33 | -0.005386758741411639, 34 | -0.010286980700035711, 35 | -0.009476857566307942, 36 | -0.0005716445337012121, 37 | 0.013060616663050698, 38 | 0.02267306167010209, 39 | 0.019259785517254543, 40 | 0.000723376637842406, 41 | -0.024191733749048327, 42 | -0.03938769356521361, 43 | -0.0315583525496071, 44 | -0.0007302161019150122, 45 | 0.036810191640485275, 46 | 0.05697731369689379, 47 | 0.043521415665204635, 48 | 0.0005472992042452829, 49 | -0.04758718282764343, 50 | -0.07050099792957325, 51 | -0.051611335733147364, 52 | -0.0002035216105500873, 53 | 0.05314581450454528, 54 | 0.07558936092565023, 55 | 0.05314581450454528, 56 | -0.0002035216105500873, 57 | -0.051611335733147364, 58 | -0.07050099792957325, 59 | -0.04758718282764343, 60 | 0.0005472992042452829, 61 | 0.043521415665204635, 62 | 0.05697731369689379, 63 | 0.036810191640485275, 64 | -0.0007302161019150122, 65 | -0.0315583525496071, 66 | -0.03938769356521361, 67 | -0.024191733749048327, 68 | 0.000723376637842406, 69 | 0.019259785517254543, 70 | 0.02267306167010209, 71 | 0.013060616663050698, 72 | -0.0005716445337012121, 73 | -0.009476857566307942, 74 | -0.010286980700035711, 75 | -0.005386758741411639, 76 | 0.0003696228529220283, 77 | 0.0034335420218244736, 78 | 0.00324942874567572, 79 | 0.0014455418248231421, 80 | -0.00007789338800231479, 81 | -0.0008646866700221276)) 82 | 83 | # Pass 0-50Hz Stop 100-1000Hz 84 | # Stop band rejection -50dB specified, got -66dB 85 | # Passband ripple 5dB specified, got 0.6dB 86 | 87 | coeffs_0 = array('f', ( 88 | -0.0000722153288409739, 89 | 0.0003137058058630565, 90 | 0.00038569220273319, 91 | 0.0005749003425041003, 92 | 0.0008280425734109133, 93 | 0.0011346221296965157, 94 | 0.0014887378298292999, 95 | 0.0018814765248971652, 96 | 0.002299518125800201, 97 | 0.0027248838317960747, 98 | 0.003134965231304449, 99 | 0.0035031351253230567, 100 | 0.0037996986161231254, 101 | 0.003993157175022867, 102 | 0.004051893948396048, 103 | 0.003946112840593105, 104 | 0.003649969034869255, 105 | 0.0031437941879818064, 106 | 0.002416242838343031, 107 | 0.0014662999820256536, 108 | 0.000305034184804234, 109 | -0.0010430492122074377, 110 | -0.0025392466792292555, 111 | -0.004130678277876215, 112 | -0.005751049775881588, 113 | -0.007322155703621066, 114 | -0.008756130180246412, 115 | -0.009958330410979013, 116 | -0.010831073488386338, 117 | -0.011278072334255495, 118 | -0.011208838308213781, 119 | -0.01054327350041362, 120 | -0.009216350964953384, 121 | -0.007181670621633992, 122 | -0.00441622494402543, 123 | -0.0009234713204266227, 124 | 0.003267606213602425, 125 | 0.008097703551613553, 126 | 0.013480668331577402, 127 | 0.01930425810934074, 128 | 0.02543307750077006, 129 | 0.03171306481322245, 130 | 0.03797652147310733, 131 | 0.04404809796449255, 132 | 0.04975147197554726, 133 | 0.05491606886755329, 134 | 0.059383735186375505, 135 | 0.0630151042478662, 136 | 0.06569528478024036, 137 | 0.0673386481774585, 138 | 0.06789238429813454, 139 | 0.0673386481774585, 140 | 0.06569528478024036, 141 | 0.0630151042478662, 142 | 0.059383735186375505, 143 | 0.05491606886755329, 144 | 0.04975147197554726, 145 | 0.04404809796449255, 146 | 0.03797652147310733, 147 | 0.03171306481322245, 148 | 0.02543307750077006, 149 | 0.01930425810934074, 150 | 0.013480668331577402, 151 | 0.008097703551613553, 152 | 0.003267606213602425, 153 | -0.0009234713204266227, 154 | -0.00441622494402543, 155 | -0.007181670621633992, 156 | -0.009216350964953384, 157 | -0.01054327350041362, 158 | -0.011208838308213781, 159 | -0.011278072334255495, 160 | -0.010831073488386338, 161 | -0.009958330410979013, 162 | -0.008756130180246412, 163 | -0.007322155703621066, 164 | -0.005751049775881588, 165 | -0.004130678277876215, 166 | -0.0025392466792292555, 167 | -0.0010430492122074377, 168 | 0.000305034184804234, 169 | 0.0014662999820256536, 170 | 0.002416242838343031, 171 | 0.0031437941879818064, 172 | 0.003649969034869255, 173 | 0.003946112840593105, 174 | 0.004051893948396048, 175 | 0.003993157175022867, 176 | 0.0037996986161231254, 177 | 0.0035031351253230567, 178 | 0.003134965231304449, 179 | 0.0027248838317960747, 180 | 0.002299518125800201, 181 | 0.0018814765248971652, 182 | 0.0014887378298292999, 183 | 0.0011346221296965157, 184 | 0.0008280425734109133, 185 | 0.0005749003425041003, 186 | 0.00038569220273319, 187 | 0.0003137058058630565, 188 | -0.0000722153288409739)) 189 | -------------------------------------------------------------------------------- /non_realtime/correlate.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/peterhinch/micropython-filters/4e0b737574073bab36ec1c776e8dfb80b8fe5f9f/non_realtime/correlate.jpg -------------------------------------------------------------------------------- /non_realtime/correlate.py: -------------------------------------------------------------------------------- 1 | # correlate.py Demo of retrieving signal buried in noise 2 | 3 | # Released under the MIT licence. 4 | # Copyright Peter Hinch 2018 5 | 6 | # This simulates a Pyboard ADC acquiring a signal using read_timed. The signal is 7 | # digital (switching between two fixed voltages) but is corrupted by a larger analog 8 | # noise source. 9 | # The output array contains the correlation of the corrupted input with the known 10 | # expected signal. 11 | # A successful detection comprises a uniquely large value at sample 930 where the 12 | # end of the signal is located. The program also outputs the second largest 13 | # correlation value and the ratio of the two as a measure of the certainty of 14 | # detection. 15 | 16 | # The chosen signal is a pseudo-random digital burst. This was chosen because the 17 | # auto-correlation function of random noise is an impulse function. 18 | # The module autocorrelate.py was used to generate this 19 | 20 | # In this test it is added to a pseudo-random analog signal of much longer 21 | # duration to test the reliability of discriminaton. 22 | 23 | import utime 24 | import pyb 25 | from array import array 26 | from filt import dcf, WRAP, SCALE, REVERSE 27 | # Read buffer length 28 | RBUFLEN = 1000 29 | # Digital amplitude. Compares with analog amplitude of 1000. 30 | DIGITAL_AMPLITUDE = 600 # 400 is about the limit with occasional false positives 31 | 32 | def rn_analog(): # Random number in range +- 1000 33 | return int(pyb.rng() / 536870 - 1000) 34 | 35 | # Max runlength 2, DR == 10 (5 bits/50) 36 | # in 100 runs lowest DR seen was 1.6, with no false positives. 37 | signal = bytearray([1,-1,1,-1,1,-1,1,1,-1,-1,1,1,-1,-1,1,-1, 38 | 1,-1,1,1,-1,1,-1,1,-1,1,-1,-1,1,1,-1,1, 39 | -1,-1,1,-1,1,-1,1,1,-1,-1,1,-1,1,1,-1,1, 40 | -1,-1,]) 41 | siglen = len(signal) 42 | 43 | # Input buffer contains random noise. While this is a simulation, the bias of 2048 44 | # emulates a Pyboard ADC biassed at its mid-point (1.65V). 45 | # Order (as from read_timed) is oldest first. 46 | bufin = array('H', (2048 + rn_analog() for i in range(RBUFLEN))) # range 2048 +- 1000 47 | 48 | # Add signal in. Burst ends 20 samples before the end. 49 | x = RBUFLEN - siglen - 20 50 | for s in signal: 51 | s = 1 if s == 1 else -1 52 | bufin[x] += s * DIGITAL_AMPLITUDE 53 | x += 1 54 | 55 | # Coeffs hold the expected signal in normal time order (oldest first). 56 | coeffs = array('f', (1 if signal[i] == 1 else -1 for i in range(siglen))) # range +-1 57 | 58 | op = array('f', (0 for _ in range(RBUFLEN))) 59 | setup = array('i', [0]*5) 60 | setup[0] = len(bufin) 61 | setup[1] = len(coeffs) 62 | setup[2] = SCALE # No wrap, normal time order. No copy back: I/P sample set unchanged 63 | setup[3] = 1 # No decimation. 64 | setup[4] = 2048 # Offset 65 | op[0] = 0.001 # Scale 66 | t = utime.ticks_us() 67 | n_results = dcf(bufin, op, coeffs, setup) 68 | t = utime.ticks_diff(utime.ticks_us(), t) 69 | 70 | ns = 0 71 | maxop = 0 72 | for x in range(n_results): 73 | res = op[x] 74 | print('{:3d} {:8.1f}'.format(x, res)) 75 | if res > maxop: 76 | maxop = res 77 | ns = x # save sample no. 78 | nextop = 0 79 | for x in op: 80 | if x < maxop and x > nextop: 81 | nextop = x 82 | s = 'Max correlation {:5.1f} at sample {:d} Next largest {:5.1f} Detection ratio {:5.1f}.' 83 | print(s.format(maxop, ns, nextop, maxop/nextop)) 84 | if ns == 930: 85 | print('Correct detection.') 86 | else: 87 | print('FALSE POSITIVE.') 88 | print('Duration {:5d}μs'.format(t)) 89 | -------------------------------------------------------------------------------- /non_realtime/filt.py: -------------------------------------------------------------------------------- 1 | # filt.py Perform FIR filtering (discrete convolution) on a 16-bit integer 2 | # array of samples. 3 | # Author: Peter Hinch 4 | # 25th March 2018 5 | 6 | # The MIT License (MIT) 7 | # 8 | # Copyright (c) 2018 Peter Hinch 9 | # 10 | # Permission is hereby granted, free of charge, to any person obtaining a copy 11 | # of this software and associated documentation files (the "Software"), to deal 12 | # in the Software without restriction, including without limitation the rights 13 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 14 | # copies of the Software, and to permit persons to whom the Software is 15 | # furnished to do so, subject to the following conditions: 16 | # 17 | # The above copyright notice and this permission notice shall be included in 18 | # all copies or substantial portions of the Software. 19 | # 20 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 21 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 22 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 23 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 24 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 25 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 26 | # THE SOFTWARE. 27 | 28 | # Register usage 29 | # r0, r1, r2 are used as variable pointers into their arrays. 30 | # r0 sample set, halfword array for dcf, float for dcf_fp 31 | # r1 result array, float, same length as sample set. Initially r[0] is scaling factor (default 1) 32 | # r2 coeffs, float. 33 | # r3 setup, int array [iplen, coeff_len, flags, decimate, offset] If decimate == 0 don't copy back to I/P 34 | # After initial setup r3 is scratch. 35 | # r4 scratch 36 | # r5 coeff counter 37 | # r6 cidx: index into samples as coeffs are processed. 38 | # r7 addr of sample set start 39 | # r8 no. of samples 40 | # r9 addr of coeffs 41 | # r10 Point to one sample after end of sample set 42 | # r11 Flags 43 | # r12 No. of coeffs 44 | # s0 scaling factor 45 | # s1 mean 46 | # s2 current sample 47 | # s3 res 48 | # s4 current coeff 49 | # s5 0.0 50 | # s6 Coeff inc/dec amount (+4 or -4) 51 | # s7 No. of samples to process (decrements) 52 | # s8 Start of result array 53 | # s9 Bytes to decrement sample pointer 54 | # s10 No. of samples to process (constant) 55 | # Flag bits. 56 | WRAP = const(1) 57 | SCALE = const(2) 58 | REVERSE = const(4) 59 | COPY = const(8) 60 | 61 | @micropython.asm_thumb 62 | def dcf(r0, r1, r2, r3): 63 | push({r8, r9, r10, r11, r12}) 64 | # Populate registers 65 | vmov(s8, r1) 66 | ldr(r4, [r3, 8]) 67 | mov(r11, r4) # R11 = Flags 68 | mov(r7, SCALE) 69 | tst(r4, r7) 70 | ittte(eq) 71 | mov(r5, 1) # No scale factor: s0 = 1.0 72 | vmov(s0, r5) 73 | vcvt_f32_s32(s0, s0) 74 | vldr(s0, [r1, 0]) # S0 = scaling factor from op array 75 | 76 | mov(r4, 0) 77 | vmov(s5, r4) 78 | vcvt_f32_s32(s5, s5) # S5 = 0.0 79 | 80 | mov(r7, r0) # R7 -> sample set start 81 | mov(r9, r2) # R9 -> coeff start 82 | 83 | ldr(r4, [r3, 0]) 84 | mov(r8, r4) # R8 = no. of samples 85 | add(r4, r4, r4) # 2 bytes per sample 86 | add(r0, r0, r4) 87 | mov(r10, r0) # R10 -> one sample after end of sample set 88 | 89 | ldr(r4, [r3, 4]) 90 | mov(r12, r4) # R12 = no. of coeffs 91 | sub(r4, 1) 92 | add(r4, r4, r4) 93 | add(r4, r4, r4) 94 | add(r2, r2, r4) # r2 -> last coeff 95 | mov(r5, 4) # Amount to inc/dec coeff pointer 96 | mov(r4, r11) # Flags 97 | mov(r6, REVERSE) 98 | tst(r4, r6) 99 | itt(eq) # If REVERSE not set, apply coeffs most recent first. 100 | mov(r9, r2) # R9 -> last (most recent) coeff 101 | neg(r5, r5) 102 | vmov(s6, r5) 103 | 104 | mov(r5, r8) # No of samples 105 | mov(r6, WRAP) 106 | tst(r4, r6) 107 | ittt(eq) # If wrapping, r5 == no. of iterations 108 | mov(r6, r12) # Not wrapping: No. of coeffs 109 | sub(r5, r5, r6) # If no wrap, no. to process = samples - coeffs + 1 110 | add(r5, 1) 111 | 112 | ldr(r4, [r3, 12]) # Decimate ( >= 1) 113 | cmp(r4, 1) 114 | it(mi) 115 | mov(r4, 1) # Avoid crash if user set decimate == 0 116 | udiv(r5, r5, r4) 117 | vmov(s7, r5) # No. of samples to process 118 | vmov(s10, r5) 119 | 120 | add(r4, r4, r4) # 2 bytes per sample 121 | vmov(s9, r4) # Bytes to decrement sample pointer 122 | 123 | ldr(r4, [r3, 16]) # Offset 124 | cmp(r4, 0) 125 | bmi(DOMEAN) # -ve offset: calculate mean 126 | vmov(s1, r4) # offset >= 0 127 | vcvt_f32_s32(s1, s1) # S1 = provided offset 128 | b(DO_FILT) 129 | 130 | # CALCULATE MEAN 131 | label(DOMEAN) 132 | # Point r0 to one after last sample 133 | vmov(r4, s5) # Zero mean 134 | vmov(s1, r4) 135 | label(MEAN) 136 | sub(r0, 2) 137 | # get 16 bit sample and 0-extend. Note this works only for samples >= 0 138 | ldrh(r4, [r0, 0]) 139 | vmov(s2, r4) 140 | vcvt_f32_s32(s2, s2) 141 | vadd(s1, s1, s2) 142 | cmp(r0, r7) # sample set start 143 | bne(MEAN) 144 | vmov(s2, r8) # no. of samples 145 | vcvt_f32_s32(s2, s2) # convert to float 146 | vdiv(s1, s1, s2) 147 | 148 | # MEAN IS NOW IN S1 149 | label(DO_FILT) 150 | mov(r0, r10) 151 | sub(r0, 2) # R0 -> last (most recent) sample 152 | vmov(r4, s7) # No. of results == no. of samples to be processed 153 | add(r4, r4, r4) # 4 bytes per result 154 | add(r4, r4, r4) 155 | add(r1, r1, r4) # R1: one after last result 156 | 157 | label(SAMPLE) 158 | sub(r1, 4) # Point to result array 159 | # get 16 bit sample and 0-extend. Note this works only for samples >= 0 160 | ldrh(r4, [r0, 0]) 161 | vmov(s2, r4) 162 | vcvt_f32_s32(s2, s2) 163 | 164 | vmov(r4, s5) # Zero result register S3 165 | vmov(s3, r4) 166 | # Set R2 coeff pointer to start / end and put update value in R3 167 | mov(r2, r9) # R2 -> coeff[0] or coeff[last] depending on REVERSE 168 | vmov(r3, s6) # Coeff pointer will inc/dec by 4 169 | 170 | mov(r6, r0) # Current sample cidx 171 | mov(r5, r12) # no. of coeffs 172 | label(COEFF) 173 | vldr(s4, [r2, 0]) # get current coeff and point to next 174 | add(r2, r2, r3) # (inc or dec by 4) 175 | 176 | # get 16 bit sample and 0-extend. Note this works only for samples >= 0 177 | ldrh(r4, [r6, 0]) 178 | vmov(s2, r4) 179 | vcvt_f32_s32(s2, s2) # Float sample value in s2 180 | vsub(s2, s2, s1) # Subtract mean 181 | vmul(s2, s2, s4) # Multiply coeff 182 | vadd(s3, s3, s2) # Add into result 183 | sub(r6, 2) # Point to next oldest sample 184 | 185 | cmp(r6, r7) # OK if current >= start 186 | bge(SKIP) 187 | # R6 -> before start 188 | mov(r6, r10) # R6 -> one after sample set end 189 | sub(r6, 2) # R6 -> newest sample 190 | 191 | label(SKIP) 192 | sub(r5, 1) # Decrement coeff counter 193 | bne(COEFF) 194 | 195 | vmul(s3, s3, s0) # Scale 196 | vstr(s3, [r1, 0]) # Store in result array 197 | 198 | vmov(r4, s9) # Bytes to decrement sample pointer 199 | sub(r0, r0, r4) 200 | 201 | vmov(r4, s7) 202 | sub(r4, 1) 203 | vmov(s7, r4) 204 | bne(SAMPLE) 205 | 206 | mov(r4, r11) # Flags 207 | mov(r6, COPY) 208 | tst(r4, r6) 209 | beq(END) # No copy back to source 210 | 211 | # COPY BACK 212 | vmov(r1, s8) # r1-> start of result array 213 | mov(r0, r7) # r0 -> start of sample array 214 | vmov(r4, s10) # No. of samples which were processed 215 | label(COPY_LOOP) 216 | vldr(s3, [r1, 0]) # Current result 217 | vadd(s3, s3, s1) # Restore mean 218 | vcvt_s32_f32(s3, s3) # Convert to integer 219 | vmov(r6, s3) 220 | strh(r6, [r0, 0]) # Store in IP array 221 | add(r0, 2) 222 | add(r1, 4) # Next result 223 | sub(r4, 1) 224 | bgt(COPY_LOOP) 225 | vcvt_s32_f32(s3, s1) # get mean 226 | vmov(r6, s3) 227 | mov(r4, r10) # 1 sample after end 228 | label(COPY_LOOP_1) # Set any uncomputed elements to mean 229 | cmp(r0, r4) 230 | bpl(END) 231 | strh(r6, [r0, 0]) # Store in IP array 232 | add(r0, 2) 233 | b(COPY_LOOP_1) 234 | 235 | label(END) 236 | pop({r8, r9, r10, r11, r12}) 237 | vmov(r0, s10) 238 | 239 | # Version where r0 -> array of float input samples 240 | @micropython.asm_thumb 241 | def dcf_fp(r0, r1, r2, r3): 242 | push({r8, r9, r10, r11, r12}) 243 | # Populate registers 244 | vmov(s8, r1) 245 | ldr(r4, [r3, 8]) 246 | mov(r11, r4) # R11 = Flags 247 | mov(r7, SCALE) 248 | tst(r4, r7) 249 | ittte(eq) 250 | mov(r5, 1) # No scale factor: s0 = 1.0 251 | vmov(s0, r5) 252 | vcvt_f32_s32(s0, s0) 253 | vldr(s0, [r1, 0]) # S0 = scaling factor from op array 254 | 255 | mov(r4, 0) 256 | vmov(s5, r4) 257 | vcvt_f32_s32(s5, s5) # S5 = 0.0 258 | 259 | mov(r7, r0) # R7 -> sample set start 260 | mov(r9, r2) # R9 -> coeff start 261 | 262 | ldr(r4, [r3, 0]) 263 | mov(r8, r4) # R8 = no. of samples 264 | add(r4, r4, r4) # 4 bytes per sample 265 | add(r4, r4, r4) 266 | add(r0, r0, r4) 267 | mov(r10, r0) # R10 -> one sample after end of sample set 268 | 269 | ldr(r4, [r3, 4]) 270 | mov(r12, r4) # R12 = no. of coeffs 271 | sub(r4, 1) 272 | add(r4, r4, r4) 273 | add(r4, r4, r4) 274 | add(r2, r2, r4) # r2 -> last coeff 275 | mov(r5, 4) # Amount to inc/dec coeff pointer 276 | mov(r4, r11) # Flags 277 | mov(r6, REVERSE) 278 | tst(r4, r6) 279 | itt(eq) # If REVERSE not set, apply coeffs most recent first. 280 | mov(r9, r2) # R9 -> last (most recent) coeff 281 | neg(r5, r5) 282 | vmov(s6, r5) 283 | 284 | mov(r5, r8) # No of samples 285 | mov(r6, WRAP) 286 | tst(r4, r6) 287 | ittt(eq) # If wrapping, r5 == no. of iterations 288 | mov(r6, r12) # Not wrapping: No. of coeffs 289 | sub(r5, r5, r6) # If no wrap, no. to process = samples - coeffs + 1 290 | add(r5, 1) 291 | 292 | ldr(r4, [r3, 12]) # Decimate ( >= 1) 293 | cmp(r4, 1) 294 | it(mi) 295 | mov(r4, 1) # Avoid crash if user set decimate == 0 296 | udiv(r5, r5, r4) 297 | vmov(s7, r5) # No. of samples to process 298 | vmov(s10, r5) 299 | 300 | add(r4, r4, r4) # 4 bytes per sample 301 | add(r4, r4, r4) 302 | vmov(s9, r4) # Bytes to decrement sample pointer 303 | 304 | ldr(r4, [r3, 16]) # Offset 305 | cmp(r4, 0) 306 | bmi(DOMEAN) # -ve offset: calculate mean 307 | vmov(s1, r4) # offset >= 0 308 | vcvt_f32_s32(s1, s1) # S1 = provided offset 309 | b(DO_FILT) 310 | 311 | # CALCULATE MEAN 312 | label(DOMEAN) 313 | # R0 points to one after last sample 314 | vmov(r4, s5) # Zero mean 315 | vmov(s1, r4) 316 | label(MEAN) 317 | sub(r0, 4) 318 | # get sample 319 | vldr(s2, [r0, 0]) 320 | vadd(s1, s1, s2) 321 | cmp(r0, r7) # sample set start 322 | bne(MEAN) 323 | vmov(s2, r8) # no. of samples 324 | vcvt_f32_s32(s2, s2) # convert to float 325 | vdiv(s1, s1, s2) 326 | 327 | # MEAN IS NOW IN S1 328 | label(DO_FILT) 329 | mov(r0, r10) 330 | sub(r0, 4) # R0 -> last (most recent) sample 331 | vmov(r4, s7) # No. of results == no. of samples to be processed 332 | add(r4, r4, r4) # 4 bytes per result 333 | add(r4, r4, r4) 334 | add(r1, r1, r4) # R1: one after last result 335 | 336 | label(SAMPLE) 337 | sub(r1, 4) # Point to result array 338 | # get sample 339 | vldr(s2, [r0, 0]) 340 | 341 | vmov(r4, s5) # Zero result register S3 342 | vmov(s3, r4) 343 | # Set R2 coeff pointer to start / end and put update value in R3 344 | mov(r2, r9) # R2 -> coeff[0] or coeff[last] depending on REVERSE 345 | vmov(r3, s6) # Coeff pointer will inc/dec by 4 346 | 347 | mov(r6, r0) # Current sample cidx 348 | mov(r5, r12) # no. of coeffs 349 | label(COEFF) 350 | vldr(s4, [r2, 0]) # get current coeff and point to next 351 | add(r2, r2, r3) # (inc or dec by 4) 352 | 353 | vldr(s2, [r6, 0]) # get current sample 354 | vsub(s2, s2, s1) # Subtract mean 355 | vmul(s2, s2, s4) # Multiply coeff 356 | vadd(s3, s3, s2) # Add into result 357 | sub(r6, 4) # Point to next oldest sample 358 | 359 | cmp(r6, r7) # OK if current >= start 360 | bge(SKIP) 361 | # R6 -> before start 362 | mov(r6, r10) # R6 -> one after sample set end 363 | sub(r6, 4) # R6 -> newest sample 364 | 365 | label(SKIP) 366 | sub(r5, 1) # Decrement coeff counter 367 | bne(COEFF) 368 | 369 | vmul(s3, s3, s0) # Scale 370 | vstr(s3, [r1, 0]) # Store in result array 371 | 372 | vmov(r4, s9) # Bytes to decrement sample pointer 373 | sub(r0, r0, r4) 374 | 375 | vmov(r4, s7) 376 | sub(r4, 1) 377 | vmov(s7, r4) 378 | bne(SAMPLE) 379 | 380 | mov(r4, r11) # Flags 381 | mov(r6, COPY) 382 | tst(r4, r6) 383 | beq(END) # No copy back to source 384 | 385 | # COPY BACK 386 | vmov(r1, s8) # r1-> start of result array 387 | mov(r0, r7) # r0 -> start of sample array 388 | vmov(r4, s10) # No. of samples which were processed 389 | label(COPY_LOOP) 390 | vldr(s3, [r1, 0]) # Current result 391 | vadd(s3, s3, s1) # Restore mean 392 | vstr(s3, [r0, 0]) # Store 393 | add(r0, 4) 394 | add(r1, 4) # Next result 395 | sub(r4, 1) 396 | bgt(COPY_LOOP) 397 | mov(r4, r10) # 1 sample after end 398 | label(COPY_LOOP_1) # Set any uncomputed elements to mean 399 | cmp(r0, r4) 400 | bpl(END) 401 | vstr(s1, [r0, 0]) # Store mean in I/P array 402 | add(r0, 4) 403 | b(COPY_LOOP_1) 404 | 405 | label(END) 406 | pop({r8, r9, r10, r11, r12}) 407 | vmov(r0, s10) 408 | -------------------------------------------------------------------------------- /non_realtime/filt_test.py: -------------------------------------------------------------------------------- 1 | # filt_test.py Test/demo program for filt.py 2 | 3 | # Released under the MIT licence. 4 | # Copyright Peter Hinch 2018 5 | 6 | # This program simulates the acquisition of an array of samples on a Pyboard ADC 7 | # using read_timed. The received signal is a sinewave biassed to lie in the 8 | # middle of the ADC voltage range. Two FIR filters are applied. 9 | # The first is a low pass filter which extracts the DC bias level. 10 | # The second is a bandpass filter which extracts the signal. The passband is 11 | # centred on a frequency corresponding to 8 cycles in the input buffer. The 12 | # rejection characteristics may be demonstrated by altering the value of 13 | # NCYCLES - try a value of 6 or 12. 14 | # See coeffs.py for bandpass filter characteristics. 15 | 16 | from array import array 17 | from coeffs import coeffs_8a, coeffs_0 18 | from math import sin, pi 19 | import utime 20 | from filt import dcf, WRAP, SCALE, REVERSE, COPY 21 | 22 | RBUFLEN = 128 23 | # Number of cycles in input buffer. 8 == centre of passband of coeffs_8a. 24 | NCYCLES = 8 25 | cycles = RBUFLEN / NCYCLES 26 | 27 | bufin = array('H', (2048 + int(1500 * sin(2 * cycles * pi * i / RBUFLEN)) for i in range(RBUFLEN))) 28 | setup = array('i', (0 for _ in range(5))) 29 | op = array('f', (0 for _ in range(RBUFLEN))) 30 | 31 | # This filter extracts a DC level in presence of noise 32 | # Wrap because signal sought (DC) is constant. 33 | setup[0] = len(bufin) 34 | setup[1] = len(coeffs_0) 35 | setup[2] = WRAP | SCALE 36 | setup[3] = 1 # Decimate by 1 37 | setup[4] = 0 # zero offset 38 | op[0] = 0.967 # Scale 39 | dcf(bufin, op, coeffs_0, setup) 40 | print('DC', sum(op)/len(op)) 41 | 42 | # This is a bandpass filter centred on a frequency such that the 43 | # input sample array contains 8 full cycles 44 | setup[0] = len(bufin) 45 | setup[1] = len(coeffs_8a) 46 | setup[2] = SCALE | COPY 47 | setup[3] = 1 # Decimate 48 | setup[4] = -1 # offset == -1: calculate mean 49 | op[0] = 1.037201 50 | 51 | t = utime.ticks_us() 52 | n_results = dcf(bufin, op, coeffs_8a, setup) 53 | t = utime.ticks_diff(utime.ticks_us(), t) 54 | print('No. of results =', n_results) 55 | 56 | print('Sample O/P input buffer') 57 | for i in range(n_results): 58 | print('{:3d} {:8.1f} {:5d}'.format(i, op[i], bufin[i])) 59 | print('Duration {:5d}μs'.format(t)) 60 | 61 | 62 | -------------------------------------------------------------------------------- /non_realtime/filt_test_all.py: -------------------------------------------------------------------------------- 1 | # filt_test_all.py 2 | # Test suite for filt.py non-realtime inline assembler routines. 3 | # Run on Pyboard. 4 | 5 | # Released under the MIT licence. 6 | # Copyright Peter Hinch 2018 7 | 8 | from array import array 9 | from filt import dcf, dcf_fp, WRAP, SCALE, REVERSE, COPY 10 | SIGLEN = const(32) 11 | 12 | # Setup common to all tests 13 | setup = array('i', (0 for _ in range(5))) 14 | setup[0] = 2 * SIGLEN # Input buffer length 15 | setup[1] = SIGLEN # Coefficient buffer length 16 | 17 | # Basic test 18 | def test1(): 19 | exp = [0.0,-1.0,-2.0,-3.0,-4.0,-5.0,-6.0,-7.0,-8.0,-9.0,-10.0,-11.0,-12.0,-13.0,-14.0, 20 | -15.0,-16.0,-13.0,-10.0,-7.0,-4.0,-1.0,2.0,5.0,8.0,11.0,14.0,17.0,20.0,23.0,26.0, 21 | 29.0,32.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0, 22 | 0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0] 23 | 24 | setup[2] = 0 # Normal time order. No wrap. 25 | setup[3] = 1 # No decimation 26 | setup[4] = 0 # Offset 27 | bufin = array('f', (0 for i in range(2*SIGLEN))) 28 | op = array('f', (0 for _ in range(2*SIGLEN))) 29 | coeffs = array('f', (-1 if x < SIGLEN/2 else 1 for x in range(SIGLEN))) 30 | idx = SIGLEN 31 | for coeff in coeffs: 32 | bufin[idx] = coeff 33 | idx += 1 34 | n_results = dcf_fp(bufin, op, coeffs, setup) 35 | ok = True 36 | for idx, x in enumerate(op): 37 | if x != exp[idx]: 38 | print('FAIL') 39 | ok = False 40 | break 41 | if n_results != SIGLEN + 1: 42 | print('Siglen fail') 43 | ok = False 44 | return ok 45 | 46 | # Test Decimation 47 | def test2(): 48 | exp = [-4.0,-8.0,-12.0,-16.0,-4.0,8.0,20.0,32.0,0.0,0.0, 49 | 0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0, 50 | 0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0, 51 | 0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0, 52 | 0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0, 53 | 0.0,0.0,0.0,0.0,0.0,0.0,] 54 | 55 | setup[2] = 0 # Normal time order. No wrap. 56 | setup[3] = 4 # Decimation by 4 57 | setup[4] = 0 # No offset 58 | bufin = array('f', (0 for i in range(2*SIGLEN))) 59 | op = array('f', (0 for _ in range(2*SIGLEN))) 60 | coeffs = array('f', (-1 if x < SIGLEN/2 else 1 for x in range(SIGLEN))) 61 | idx = SIGLEN 62 | for coeff in coeffs: 63 | bufin[idx] = coeff 64 | idx += 1 65 | n_results = dcf_fp(bufin, op, coeffs, setup) 66 | ok = True 67 | for idx, x in enumerate(op): 68 | if x != exp[idx]: 69 | print('FAIL') 70 | ok = False 71 | break 72 | if n_results != SIGLEN // 4: # Decimation factor 73 | print('Siglen fail') 74 | ok = False 75 | return ok 76 | 77 | # Test copy 78 | def test3(): 79 | exp = [0.0,-1.0,-2.0,-3.0,-4.0,-5.0,-6.0,-7.0,-8.0,-9.0,-10.0,-11.0,-12.0,-13.0,-14.0, 80 | -15.0,-16.0,-13.0,-10.0,-7.0,-4.0,-1.0,2.0,5.0,8.0,11.0,14.0,17.0,20.0,23.0,26.0, 81 | 29.0,32.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0, 82 | 0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0] 83 | 84 | setup[2] = COPY # Normal time order. Copy. 85 | setup[3] = 1 # No decimation 86 | setup[4] = 0 # No offset 87 | bufin = array('f', (0 for i in range(2*SIGLEN))) 88 | op = array('f', (0 for _ in range(2*SIGLEN))) 89 | coeffs = array('f', (-1 if x < SIGLEN/2 else 1 for x in range(SIGLEN))) 90 | idx = SIGLEN 91 | for coeff in coeffs: 92 | bufin[idx] = coeff 93 | idx += 1 94 | n_results = dcf_fp(bufin, op, coeffs, setup) 95 | ok = True 96 | for idx, x in enumerate(op): 97 | if x != exp[idx]: 98 | print('FAIL') 99 | ok = False 100 | break 101 | if n_results != SIGLEN + 1: 102 | print('Siglen fail') 103 | ok = False 104 | return ok 105 | 106 | # Test wrap 107 | def test4(): 108 | exp = [29.0,26.0,23.0,20.0,17.0,14.0,11.0,8.0,5.0,2.0,-1.0,-4.0,-7.0,-10.0, 109 | -13.0,-16.0,-15.0,-14.0,-13.0,-12.0,-11.0,-10.0,-9.0,-8.0,-7.0,-6.0, 110 | -5.0,-4.0,-3.0,-2.0,-1.0,0.0,-1.0,-2.0,-3.0,-4.0,-5.0,-6.0,-7.0,-8.0, 111 | -9.0,-10.0,-11.0,-12.0,-13.0,-14.0,-15.0,-16.0,-13.0,-10.0,-7.0,-4.0, 112 | -1.0,2.0,5.0,8.0,11.0,14.0,17.0,20.0,23.0,26.0,29.0,32.0,] 113 | 114 | setup[2] = WRAP 115 | setup[3] = 1 # No decimation 116 | setup[4] = 0 # No offset 117 | bufin = array('f', (0 for i in range(2*SIGLEN))) 118 | op = array('f', (0 for _ in range(2*SIGLEN))) 119 | coeffs = array('f', (-1 if x < SIGLEN/2 else 1 for x in range(SIGLEN))) 120 | idx = SIGLEN 121 | for coeff in coeffs: 122 | bufin[idx] = coeff 123 | idx += 1 124 | n_results = dcf_fp(bufin, op, coeffs, setup) 125 | ok = True 126 | for idx, x in enumerate(op): 127 | if x != exp[idx]: 128 | print('FAIL') 129 | ok = False 130 | break 131 | if n_results != SIGLEN * 2 : 132 | print('Siglen fail') 133 | ok = False 134 | return ok 135 | 136 | # Test reverse 137 | def test5(): 138 | exp = [0.0,1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0,16.0, 139 | 13.0,10.0,7.0,4.0,1.0,-2.0,-5.0,-8.0,-11.0,-14.0,-17.0,-20.0,-23.0,-26.0, 140 | -29.0,-32.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0, 141 | 0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,] 142 | 143 | setup[2] = REVERSE 144 | setup[3] = 1 # No decimation 145 | setup[4] = 0 # No offset 146 | bufin = array('f', (0 for i in range(2*SIGLEN))) 147 | op = array('f', (0 for _ in range(2*SIGLEN))) 148 | coeffs = array('f', (-1 if x < SIGLEN/2 else 1 for x in range(SIGLEN))) 149 | idx = SIGLEN 150 | for coeff in coeffs: 151 | bufin[idx] = coeff 152 | idx += 1 153 | n_results = dcf_fp(bufin, op, coeffs, setup) 154 | ok = True 155 | for idx, x in enumerate(op): 156 | if x != exp[idx]: 157 | print('FAIL') 158 | ok = False 159 | break 160 | if n_results != SIGLEN + 1: 161 | print('Siglen fail') 162 | ok = False 163 | return ok 164 | 165 | # ***** dcf tests ***** 166 | # Normal time order, scale and copy back 167 | def test6(): 168 | exp = [2048,2047,2046,2045,2044,2043,2042,2041,2040,2039,2038,2037,2036,2035, 169 | 2034,2033,2032,2035,2038,2041,2044,2047,2050,2053,2056,2059,2062,2065, 170 | 2068,2071,2074,2077,2080,2048,2048,2048,2048,2048,2048,2048,2048,2048, 171 | 2048,2048,2048,2048,2048,2048,2048,2048,2048,2048,2048,2048,2048,2048, 172 | 2048,2048,2048,2048,2048,2048,2048,2048] 173 | 174 | setup[2] = SCALE | COPY 175 | setup[3] = 1 # No decimation 176 | setup[4] = 2048 # Offset 177 | bufin = array('H', (2048 for i in range(2*SIGLEN))) 178 | op = array('f', (0 for _ in range(2*SIGLEN))) 179 | op[0] = 0.001 # Scaling 180 | coeffs = array('f', (-1 if x < SIGLEN/2 else 1 for x in range(SIGLEN))) 181 | idx = SIGLEN 182 | for coeff in coeffs: 183 | bufin[idx] = int(2048 + 1000 * coeff) 184 | idx += 1 185 | n_results = dcf(bufin, op, coeffs, setup) 186 | ok = True 187 | for idx, x in enumerate(bufin): 188 | if x != exp[idx]: 189 | print('FAIL') 190 | ok = False 191 | break 192 | if n_results != SIGLEN + 1: 193 | print('Siglen fail') 194 | ok = False 195 | return ok 196 | 197 | # Reverse time order 198 | def test7(): 199 | exp = [2048,2049,2050,2051,2052,2053,2054,2055,2056,2057,2058,2059,2060,2061, 200 | 2062,2063,2064,2061,2058,2055,2052,2049,2046,2043,2040,2037,2034,2031, 201 | 2028,2025,2022,2019,2016,2048,2048,2048,2048,2048,2048,2048,2048,2048, 202 | 2048,2048,2048,2048,2048,2048,2048,2048,2048,2048,2048,2048,2048,2048, 203 | 2048,2048,2048,2048,2048,2048,2048,2048] 204 | 205 | setup[2] = SCALE | COPY | REVERSE 206 | setup[3] = 1 # No decimation 207 | setup[4] = 2048 # Offset 208 | bufin = array('H', (2048 for i in range(2*SIGLEN))) 209 | op = array('f', (0 for _ in range(2*SIGLEN))) 210 | op[0] = 0.001 # Scaling 211 | coeffs = array('f', (-1 if x < SIGLEN/2 else 1 for x in range(SIGLEN))) 212 | idx = SIGLEN 213 | for coeff in coeffs: 214 | bufin[idx] = int(2048 + 1000 * coeff) 215 | idx += 1 216 | n_results = dcf(bufin, op, coeffs, setup) 217 | ok = True 218 | for idx, x in enumerate(bufin): 219 | if x != exp[idx]: 220 | print('FAIL') 221 | ok = False 222 | break 223 | if n_results != SIGLEN + 1: 224 | print('Siglen fail') 225 | ok = False 226 | return ok 227 | 228 | # Circular convolution 229 | def test8(): 230 | exp = [2077,2074,2071,2068,2065,2062,2059,2056,2053,2050,2047,2044,2041, 231 | 2038,2035,2032,2033,2034,2035,2036,2037,2038,2039,2040,2041,2042, 232 | 2043,2044,2045,2046,2047,2048,2047,2046,2045,2044,2043,2042,2041, 233 | 2040,2039,2038,2037,2036,2035,2034,2033,2032,2035,2038,2041,2044, 234 | 2047,2050,2053,2056,2059,2062,2065,2068,2071,2074,2077,2080,] 235 | 236 | setup[2] = SCALE | COPY | WRAP 237 | setup[3] = 1 # No decimation 238 | setup[4] = 2048 # Offset 239 | bufin = array('H', (2048 for i in range(2*SIGLEN))) 240 | op = array('f', (0 for _ in range(2*SIGLEN))) 241 | op[0] = 0.001 # Scaling 242 | coeffs = array('f', (-1 if x < SIGLEN/2 else 1 for x in range(SIGLEN))) 243 | idx = SIGLEN 244 | for coeff in coeffs: 245 | bufin[idx] = int(2048 + 1000 * coeff) 246 | idx += 1 247 | n_results = dcf(bufin, op, coeffs, setup) 248 | ok = True 249 | for idx, x in enumerate(bufin): 250 | if x != exp[idx]: 251 | print('FAIL') 252 | ok = False 253 | break 254 | if n_results != 2 * SIGLEN: 255 | print('Siglen fail') 256 | ok = False 257 | return ok 258 | 259 | # Decimation test 260 | def test9(): 261 | exp = [2044,2040,2036,2032,2044,2056,2068,2080,2048,2048,2048,2048,2048, 262 | 2048,2048,2048,2048,2048,2048,2048,2048,2048,2048,2048,2048,2048, 263 | 2048,2048,2048,2048,2048,2048,2048,2048,2048,2048,2048,2048,2048, 264 | 2048,2048,2048,2048,2048,2048,2048,2048,2048,2048,2048,2048,2048, 265 | 2048,2048,2048,2048,2048,2048,2048,2048,2048,2048,2048,2048,] 266 | 267 | setup[2] = SCALE | COPY 268 | setup[3] = 4 # Decimation 269 | setup[4] = -1 # Offset 270 | bufin = array('H', (2048 for i in range(2*SIGLEN))) 271 | op = array('f', (0 for _ in range(2*SIGLEN))) 272 | op[0] = 0.001 # Scaling 273 | coeffs = array('f', (-1 if x < SIGLEN/2 else 1 for x in range(SIGLEN))) 274 | idx = SIGLEN 275 | for coeff in coeffs: 276 | bufin[idx] = int(2048 + 1000 * coeff) 277 | idx += 1 278 | n_results = dcf(bufin, op, coeffs, setup) 279 | ok = True 280 | #print(n_results) 281 | #print('[', end='') 282 | #for x in op: 283 | #print('{:4.1f},'.format(x), end='') 284 | #print(']') 285 | #print('[', end='') 286 | #for x in bufin: 287 | #print('{:d},'.format(x), end='') 288 | #print(']') 289 | for idx, x in enumerate(bufin): 290 | if x != exp[idx]: 291 | print('FAIL') 292 | ok = False 293 | break 294 | if n_results != SIGLEN // 4: 295 | print('Siglen fail') 296 | ok = False 297 | return ok 298 | 299 | for n, test in enumerate([test1, test2, test3, test4, test5, test6, test7, test8, test9]): 300 | if not test(): 301 | print('Test', n +1, 'failed.') 302 | break 303 | else: 304 | print('All tests passed OK.') 305 | -------------------------------------------------------------------------------- /non_realtime/samples.py: -------------------------------------------------------------------------------- 1 | # Code produced by autocorrelate.py 2 | # Each list element comprises [[data...], detection_rato, runlength] 3 | # where runlength is the actual maximum RL in the signal. 4 | # Variables data, detection_ratio and runlength hold the values 5 | # for the best detection_ratio achieved. 6 | signal_length = 50 7 | max_run_length = 6 8 | signals = [ 9 | [[-1,-1,-1,1,1,1,1,-1,-1,-1,-1,1,-1,1,1,-1, 10 | -1,-1,1,-1,1,1,1,-1,1,-1,-1,-1,1,-1,1,1, 11 | 1,-1,1,1,-1,-1,1,1,1,-1,1,-1,-1,1,1,-1, 12 | -1,1,], 3.6, 4], 13 | [[1,1,-1,1,-1,1,-1,-1,1,1,-1,1,-1,-1,-1,-1, 14 | -1,1,1,1,1,1,1,-1,1,1,-1,-1,-1,1,1,-1, 15 | -1,1,1,1,-1,1,1,-1,1,1,-1,1,-1,-1,-1,-1, 16 | -1,-1,], 5.0, 6], 17 | [[-1,-1,1,-1,-1,1,-1,1,1,1,1,1,-1,1,-1,1, 18 | 1,1,-1,-1,1,-1,-1,-1,1,1,-1,-1,-1,-1,1,1, 19 | 1,-1,1,-1,1,1,-1,-1,1,1,-1,1,-1,-1,-1,-1, 20 | 1,1,], 5.6, 5], 21 | [[1,-1,1,-1,1,-1,1,1,-1,1,-1,-1,-1,-1,1,1, 22 | -1,-1,-1,-1,1,-1,-1,-1,-1,-1,1,1,1,-1,1,-1, 23 | -1,-1,-1,1,1,1,-1,1,1,-1,1,1,-1,1,1,1, 24 | 1,1,], 7.1, 5], 25 | [[-1,-1,-1,-1,-1,1,-1,1,1,-1,-1,1,1,1,1,-1, 26 | -1,1,1,-1,-1,-1,1,-1,1,1,-1,1,-1,1,-1,1, 27 | -1,1,-1,1,1,-1,1,-1,1,1,-1,1,-1,-1,-1,1, 28 | 1,1,], 8.3, 5], 29 | [[1,-1,-1,-1,1,-1,-1,1,1,1,-1,-1,1,-1,1,1, 30 | -1,1,-1,-1,-1,1,1,1,-1,-1,-1,-1,1,1,-1,1, 31 | -1,1,1,1,-1,1,1,1,1,1,1,-1,1,-1,-1,-1, 32 | -1,-1,], 12.5, 6], 33 | ] 34 | data, detection_ratio, runlength = signals[-1] 35 | -------------------------------------------------------------------------------- /osc.py: -------------------------------------------------------------------------------- 1 | # Demo program for FIR filter module 6th Feb 2015 2 | # Outputs a swept frequency sine wave on Dac1 3 | # Timer interrupt reads the analog input, filters it, and outputs the result on Dac 2. 4 | # Requires a link between X5 and X7 5 | 6 | import math 7 | import pyb 8 | import array 9 | from fir import fir 10 | import micropython 11 | micropython.alloc_emergency_exception_buf(100) 12 | 13 | # Define hardware 14 | dac1 = pyb.DAC(1) 15 | dac2 = pyb.DAC(2) 16 | adc = pyb.ADC("X7") 17 | tim = pyb.Timer(4, freq=2000) # Sampling freq 10KHz is about the limit 14KHz without constraint 18 | 19 | # Data for FIR filter: narrowband filter designed to isolate a 100Hz signal (@2Ksps) 20 | coeffs = array.array('i', (-22340,-3286,-2750,-1678,-135,1757,3821,5842,7586,8829,9375,9089,7912,5881,3127 21 | ,-125,-3575,-6875,-9661,-11591,-12391,-11880,-10004,-6854,-2663,2207,7292,12068,16007,18627 22 | ,19553,18560,15614,10880,4727,-2301,-9531,-16224,-21648,-25157,-26261,-24696,-20455,-13815,-5318 23 | ,4267,14010,22912,30002,34442,35623,33239,27338,18335,6989,-5657,-18372,-29853,-38858,-44338 24 | ,-45554,-42171,-34313,-22568,-7959,8167,24205,38538,49613,56146,57273,52646,42488,27594,9283 25 | ,-10722,-30452,-47888,-61165,-68769,-69705,-63621,-50867,-32488,-10138,14059,37713,58406,73939,82558 26 | ,83149,75378,59759,37601,10940,-17678,-45420,-69457,-87244,-96794,-96895,-87255,-68561,-42437,-11303 27 | ,21841,53710,81059,101009,111355,110803,99126,77217,47023,11377,-26271,-62183,-92711,-114661,-125634 28 | ,-124280,-110461,-85294,-51059,-11000,30986,70726,104192,127903,139298,136995,120977,92591,54461,10233 29 | ,-35774,-78987,-115037,-140201,-151790,-148433,-130234,-98790,-57047,-9030,40550,86764,124951,151199,162738 30 | ,158236,137946,103700,58750,7463,-45096,-93698,-133543,-160456,-171710,-166013,-143785,-107096,-59479,-5585 31 | ,49271,99638,140473,167651,178380,171491,147566,108888,59225,3455,-52909,-104270,-145507,-172488,-182485 32 | ,-174450,-149132,-108998,-57996,-1167,55864,107439,148439,174791,183876,174791,148439,107439,55864,-1167 33 | ,-57996,-108998,-149132,-174450,-182485,-172488,-145507,-104270,-52909,3455,59225,108888,147566,171491,178380 34 | ,167651,140473,99638,49271,-5585,-59479,-107096,-143785,-166013,-171710,-160456,-133543,-93698,-45096,7463 35 | ,58750,103700,137946,158236,162738,151199,124951,86764,40550,-9030,-57047,-98790,-130234,-148433,-151790 36 | ,-140201,-115037,-78987,-35774,10233,54461,92591,120977,136995,139298,127903,104192,70726,30986,-11000 37 | ,-51059,-85294,-110461,-124280,-125634,-114661,-92711,-62183,-26271,11377,47023,77217,99126,110803,111355 38 | ,101009,81059,53710,21841,-11303,-42437,-68561,-87255,-96895,-96794,-87244,-69457,-45420,-17678,10940 39 | ,37601,59759,75378,83149,82558,73939,58406,37713,14059,-10138,-32488,-50867,-63621,-69705,-68769 40 | ,-61165,-47888,-30452,-10722,9283,27594,42488,52646,57273,56146,49613,38538,24205,8167,-7959 41 | ,-22568,-34313,-42171,-45554,-44338,-38858,-29853,-18372,-5657,6989,18335,27338,33239,35623,34442 42 | ,30002,22912,14010,4267,-5318,-13815,-20455,-24696,-26261,-25157,-21648,-16224,-9531,-2301,4727 43 | ,10880,15614,18560,19553,18627,16007,12068,7292,2207,-2663,-6854,-10004,-11880,-12391,-11591 44 | ,-9661,-6875,-3575,-125,3127,5881,7912,9089,9375,8829,7586,5842,3821,1757,-135 45 | ,-1678,-2750,-3286,-22340)) 46 | 47 | # Data for FIR filter Pass (@2Ksps) 0-40Hz Stop 80Hz-> 48 | lpfcoeffs = array.array('i', (72, 47, 61, 75, 90, 105, 119, 132, 142, 149, 152, 149, 49 | 140, 125, 102, 71, 33, -12, -65, -123, -187, -254, -322, -389, -453, -511, -561, 50 | -599, -622, -628, -615, -579, -519, -435, -324, -187, -23, 165, 375, 607, 855, 51 | 1118, 1389, 1666, 1941, 2212, 2472, 2715, 2938, 3135, 3303, 3437, 3535, 3594, 52 | 3614, 3594, 3535, 3437, 3303, 3135, 2938, 2715, 2472, 2212, 1941, 1666, 1389, 53 | 1118, 855, 607, 375, 165, -23, -187, -324, -435, -519, -579, -615, -628, -622, 54 | -599, -561, -511, -453, -389, -322, -254, -187, -123, -65, -12, 33, 71, 102, 125, 55 | 140, 149, 152, 149, 142, 132, 119, 105, 90, 75, 61, 47, 72)) 56 | ncoeffs = len(coeffs) 57 | data = array.array('i', [0]*(ncoeffs +3)) # Scratchpad must be three larger than coeffs 58 | data[0] = ncoeffs 59 | data[1] = 20 60 | 61 | # Data input, filter and output 62 | def cb(timer): 63 | dac2.write(fir(data, coeffs, adc.read()) // 1000000) 64 | 65 | def cb1(timer): # For filters with 0 DC response 66 | val = fir(data, coeffs, adc.read()) // 256 # Filter amd scale 67 | dac2.write(max(0, min(255, val+128))) # Constrain, shift (no DC from bandpass) and output 68 | 69 | 70 | tim.callback(cb1) 71 | 72 | # Sweep generator 73 | def sine_sweep(start, end, mult): # Emit sinewave on DAC1 74 | buf = bytearray(100) 75 | for i in range(len(buf)): 76 | buf[i] = 128 + int(110 * math.sin(2 * math.pi * i / len(buf))) 77 | 78 | freq = start 79 | while True: 80 | dac1.write_timed(buf, int(freq) * len(buf), mode=pyb.DAC.CIRCULAR) 81 | print(freq, "Hz") 82 | pyb.delay(2500) 83 | freq *= mult 84 | if freq > end: 85 | freq = start 86 | 87 | sine_sweep(70, 200, 1.02) 88 | #sine_sweep(10, 400, 1.33) 89 | 90 | --------------------------------------------------------------------------------