├── README.md
├── LICENSE
├── archive
    ├── v7
    │   └── UTXOracle.py
    ├── v8
    │   └── UTXOracle.py
    └── start9
    │   └── start9
    │       └── UTXOracle.py
└── UTXOracle.py


/README.md:
--------------------------------------------------------------------------------
 1 | <p align="center">
 2 |   <img src="https://utxo.live/oracle/oracle_yesterday.png" alt="UTXOracle Chart" width="100%">
 3 | </p>
 4 | 
 5 | # UTXOracle
 6 | 
 7 | **UTXOracle** is a Bitcoin-native, exchange-free price oracle that calculates the market price of Bitcoin directly from the blockchain.
 8 | 
 9 | Unlike traditional oracles that rely on exchange APIs, UTXOracle identifies the most statistically probable BTC/USD exchange rate by analyzing recent transactions on-chain — no external price feeds required.
10 | 
11 | > ⚡ Pure Python. No dependencies. No assumptions. Just Bitcoin data.
12 | 
13 | ---
14 | 
15 | ## 🔍 How It Works
16 | 
17 | UTXOracle analyzes confirmed Bitcoin transactions and uses statistical clustering to isolate a "canonical" price point:
18 | - Filters out coinbase, self-spends, and spam transactions.
19 | - Focuses on economically meaningful outputs (within a dynamic BTC range).
20 | - Calculates a volume-weighted median from clustered prices across a recent window of blocks.
21 | 
22 | The result is a Bitcoin price **derived from actual usage**, not speculative trading.
23 | 
24 | ---
25 | 
26 | ## 🧠 Why It Matters
27 | 
28 | - 🛑 **Exchange Independence**: Trust the chain, not custodians.
29 | - 🔎 **Transparency**: Every price is reproducible from public block data.
30 | - 🎯 **On-Chain Signal**: Derived from organic BTC/USD activity.
31 | - 🐍 **Minimalism**: The core logic fits in a single, readable Python file.
32 | 
33 | ---
34 | 
35 | ## 📦 Getting Started
36 | 
37 | Clone the repo and run the reference script:
38 | 
39 | ```bash
40 | git clone https://github.com/Unbesteveable/UTXOracle.git
41 | cd UTXOracle
42 | python3 UTXOracle.py
43 | ```
44 | 
45 | This will connect to your local `bitcoind` node and print the current UTXOracle price.
46 | 
47 | **Requirements:**
48 | - A running Bitcoin Core node (RPC enabled)
49 | - Python 3.8+
50 | 
51 | ---
52 | 
53 | ## 🌐 Live Example
54 | 
55 | Check the live visual version of UTXOracle here:  
56 | 📺 **https://utxo.live**
57 | 
58 | - Includes historical charts and real-time YouTube stream
59 | - Based entirely on the same logic as the reference script
60 | 
61 | ---
62 | 
63 | ## 🛠 Structure
64 | 
65 | - `UTXOracle.py` – The main reference implementation
66 | - `v8/`, `v9/` – Previous algorithm versions
67 | - `start9/` – Packaging for Start9 node integration
68 | 
69 | ---
70 | 
71 | ## ⚖️ License
72 | 
73 | UTXOracle is licensed under the [Blue Oak Model License 1.0.0](./LICENSE), a permissive open-source license designed to be simple, fair, and developer-friendly.
74 | 
75 | You are free to use, modify, and distribute this software with very few restrictions.
76 | 
77 | ---
78 | 
79 | ## 🙏 Credits
80 | 
81 | Created by [@Unbesteveable](https://github.com/Unbesteveable)  
82 | Inspired by the idea that **Bitcoin's price should come from Bitcoin itself.**
83 | 
84 | ---
85 | 
86 | <p align="center">
87 |   <i>Signal from noise.</i>
88 | </p>
89 | 


--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
  1 | UTXOracle License
  2 | 
  3 | Version 1.0 — May 2025
  4 | 
  5 | This is a custom license written specifically for the UTXOracle project. It
  6 | reflects the unique nature of Bitcoin data: namely, that long-term confirmed
  7 | data can achieve consensus across nodes, while real-time or mempool-based data
  8 | inherently cannot. This license is designed to:
  9 | 
 10 | - Encourage wide, free use of UTXOracle for consensus-compatible purposes;
 11 | - Prevent confusion or misuse of the term "UTXOracle" for outputs not derived
 12 |   from consensus logic;
 13 | - Retain commercial and naming rights for live-streamed or real-time products.
 14 | 
 15 | This license is not OSI-approved, but it is written in good faith to balance
 16 | decentralization, reproducibility, and responsible innovation.
 17 | 
 18 | Section 1: Definitions
 19 | 
 20 | “UTXOracle Local” refers to the open-source software made available by the
 21 | author for calculating the 24-hour average confirmed price and the recent
 22 | 144-block window price using confirmed Bitcoin transactions.
 23 | 
 24 | “Consensus-Compatible Use” means:
 25 | - Running UTXOracle Local to generate the daily average confirmed block price
 26 |   (“UTXOracle Consensus Price”), or
 27 | - Running UTXOracle Local to generate the price from the most recent 144
 28 |   confirmed blocks (“UTXOracle Block Window Price”),
 29 | - Without modifying the filtering or averaging logic that produces those prices.
 30 | 
 31 | “Live or Real-Time Use” means:
 32 | - Using mempool data,
 33 | - Using data from fewer than 6 confirmations at the chain tip,
 34 | - Generating prices that update faster than once per confirmed block,
 35 | - Producing streamed or pushed data outputs (APIs, trading bots, etc.).
 36 | 
 37 | “The Author” refers to the creator and copyright holder of UTXOracle, reachable
 38 | via Twitter.com or x.com at @SteveSimple.
 39 | 
 40 | Section 2: Permissions (Consensus-Compatible Use)
 41 | 
 42 | 1. You are free to use, modify, distribute, and run UTXOracle Local for
 43 |    Consensus-Compatible Use without cost.
 44 | 2. You may adapt UTXOracle Local to fit your local environment (e.g., paths,
 45 |    dependencies, node RPC settings) as long as you do not alter the price-filtering
 46 |    or averaging logic.
 47 | 3. You may publish or share visualizations or outputs of UTXOracle Local for
 48 |    educational, analytical, or public benefit purposes.
 49 | 4. If you are using UTXOracle Local unmodified to generate the 24-hour average
 50 |    or the 144-block window price, you must refer to these outputs by their
 51 |    canonical names:
 52 |    - “UTXOracle Consensus Price” for the 24-hour average from confirmed blocks
 53 |    - “UTXOracle Block Window Price” for the average from the most recent 144
 54 |      confirmed blocks
 55 | 5. You may not relabel or rebrand these outputs using alternative names when the
 56 |    original UTXOracle code remains unmodified for price logic.
 57 | 6. You may not use the UTXOracle Consensus Price or UTXOracle Block Window Price
 58 |    for commercial purposes, including but not limited to financial services, paid
 59 |    dashboards, or subscription-based products, without prior written permission
 60 |    from the Author.
 61 | 7. Commercial third-party node applications may integrate UTXOracle Consensus and
 62 |    Block Window Prices into their products so long as:
 63 |    - The integration adheres to all other conditions in this Section,
 64 |    - The price logic is unmodified,
 65 |    - The outputs are clearly labeled with the canonical names.
 66 |    - A link to the UTXOracle Live stream is included in the display.
 67 | 8. Any redistribution of the UTXOracle code must retain this license in its
 68 |    entirety, including this Section and all usage restrictions.
 69 | 9. Consensus-Compatible Use includes automated or repeated execution of UTXOracle
 70 |    Local logic to recalculate the UTXOracle Block Window Price on each new
 71 |    confirmed block, provided that:
 72 |    - Only confirmed blocks are used,
 73 |    - The logic remains unmodified,
 74 |    - The outputs are labeled with their canonical names.
 75 | 
 76 | Section 3: Restrictions on Naming and Representation
 77 | 
 78 | 1. If you modify the price logic (e.g., change filtering thresholds, averaging
 79 |    methods, or block selection rules), you must not use the following terms to
 80 |    describe your output:
 81 |    - “UTXOracle Consensus Price”
 82 |    - “UTXOracle Block Window Price”
 83 |    - Any term incorporating “UTXOracle” to describe the price output
 84 | 2. These terms are reserved exclusively for outputs derived from unmodified
 85 |    consensus-compatible logic as defined by the Author.
 86 | 3. You may not use the UTXOracle name, logo, or associated branding for any fork
 87 |    or derivative project without written permission.
 88 | 
 89 | Section 4: Prohibited Use (Live or Commercial Streaming)
 90 | 
 91 | 1. You may not use UTXOracle (in whole or in part), or any derivative of it, to:
 92 |    - Generate or stream a live or real-time price feed;
 93 |    - Provide price updates more frequent than once per block;
 94 |    - Operate a trading bot, financial API, or trading-related product;
 95 |    - Offer a public-facing service under the name UTXOracle.
 96 | 2. These use cases are reserved exclusively for the Author under the name:
 97 |    - “UTXOracle Live”
 98 |    - “UTXOracle Live Price”
 99 |    - “Live On-Chain Price”
100 | 3. To discuss licensing for live or commercial usage, contact the Author.
101 | 
102 | Section 5: Trademark and Branding
103 | 
104 | The following terms are being claimed as trademarks by the Author:
105 | - UTXOracle
106 | - UTXOracle Consensus Price
107 | - UTXOracle Block Window Price
108 | - UTXOracle Live
109 | - UTXOracle Live Price
110 | - Live On-Chain Price
111 | 
112 | Use of these names, phrases, or related branding in public-facing products or
113 | services is strictly prohibited without explicit written permission.
114 | 
115 | Section 6: No Warranty
116 | 
117 | This software is provided “as is,” without warranty of any kind. The Author shall
118 | not be liable for any claims, damages, or losses resulting from its use.


--------------------------------------------------------------------------------
/archive/v7/UTXOracle.py:
--------------------------------------------------------------------------------
  1 | 
  2 | ###############################################################################  
  3 | 
  4 | #             Introduction) This is UTXOracle.py           
  5 | 
  6 | ###############################################################################  
  7 | 
  8 | 
  9 | # This python program estimates the daily USD price of bitcoin using only
 10 | # your bitcoin Core full node. It will work even while you are disconnected 
 11 | # from the internet because it only reads blocks from your machine. It does not
 12 | # save files, write cookies, or access any wallet information. It only reads 
 13 | # blocks, analyzes output patterns, and estimates a daily average a USD 
 14 | # price of bitcoin. The call to your node is the standard "bitcoin-cli". The
 15 | # date and price ranges expected to work for this version are from 2020-7-26 
 16 | # and from $10,000 to $100,000
 17 | 
 18 | print("UTXOracle version 7\n")
 19 | 
 20 | 
 21 | ###############################################################################  
 22 | 
 23 | #                        Quick Start          
 24 | 
 25 | ###############################################################################  
 26 | 
 27 | 
 28 | # 1. Make sure you have python3 and bitcoin-cli installed
 29 | # 2. Make sure "server = 1" is in bitcoin.conf
 30 | # 3. Run this file as "python3 UTXOracle.py"
 31 | 
 32 | # If this isn't working for you, you'll likely need to explore the
 33 | # bitcon-cli configuration options below:
 34 | 
 35 | 
 36 | # configuration options for bitcoin-cli
 37 | datadir      = ""
 38 | rpcuser      = "" 
 39 | rpcpassword  = ""
 40 | rpcookiefile = ""
 41 | rpcconnect   = ""
 42 | rpcport      = ""
 43 | conf         = ""
 44 | 
 45 | 
 46 | #add the configuration options to the bitcoin-cli call
 47 | bitcoin_cli_options = []
 48 | if datadir      != "":
 49 |     bitcoin_cli_options.append('-datadir='+ datadir)
 50 | if rpcuser      != "":
 51 |     bitcoin_cli_options.append("-rpcuser="+ rpcuser)
 52 | if rpcpassword  != "":
 53 |     bitcoin_cli_options.append("-rpcpassword="+ rpcpassword)
 54 | if rpcookiefile != "":
 55 |     bitcoin_cli_options.append("-rpcookiefile="+ rpcookiefile)
 56 | if rpcconnect   != "":
 57 |     bitcoin_cli_options.append("-rpcconnect="+ rpcconnect)
 58 | if rpcport      != "":
 59 |     bitcoin_cli_options.append("-rpcport="+ rpcport)
 60 | if conf         != "":
 61 |     bitcoin_cli_options.append("-conf="+ conf)
 62 | 
 63 | 
 64 | 
 65 | 
 66 | 
 67 | 
 68 | 
 69 | 
 70 | ###############################################################################  
 71 | 
 72 | #  Part 1) Defining a shortcut function to call your node      
 73 | 
 74 | ###############################################################################  
 75 | 
 76 | # Here we define define a shortcut (a function) for calling the node as we do 
 77 | # this many times through out the program. The function will return the 
 78 | # answer it gets from your node with the "command" that you asked it for.
 79 | # If we don't get an answer, the problem is likely that you don't have
 80 | # sever=1 in your bitcoin conf file.
 81 | 
 82 | 
 83 | 
 84 | import subprocess #a built in python library for sending command line commands
 85 | def Ask_Node(command):
 86 |     
 87 |     # 'bitcoin-cli' is how the command window calls your node so
 88 |     # it needs to be the first word in any request for data from the node
 89 |     # other options are added if given
 90 |     for o in bitcoin_cli_options:
 91 |         command.insert(0,o)
 92 |     command.insert(0,"bitcoin-cli")
 93 |     
 94 |     # get the answer from the node and return it to the program
 95 |     answer = None
 96 |     try:  #python try is used when we need to deal with errors after
 97 |         answer = subprocess.check_output(command)
 98 |     except Exception as e:
 99 |         # something went wrong while getting the answer
100 |         print("Error connecting to your node. Trouble shooting steps:\n")
101 |         print("\t 1) Make sure bitcoin-cli is working. Try command 'bitcoin-cli getblockcount'")
102 |         print("\t 2) Make sure config file bitcoin.conf has server=1")
103 |         print("\t 3) Explore the bitcoin-cli options in UTXOracle.py (line 38)")
104 |         print("\nThe command was:"+str(command))
105 |         print("\nThe error from bitcoin-cli was:\n")
106 |         print(e)
107 |         exit()
108 |         
109 |     # answer received, return this answer to the program
110 |     return answer
111 | 
112 | 
113 | 
114 | 
115 | 
116 | 
117 | 
118 | 
119 | 
120 | 
121 | 
122 | 
123 | ###############################################################################  
124 | 
125 | # Part 2)  Get the latest block from the node      
126 | 
127 | ###############################################################################  
128 | 
129 | # The first request to the node is to ask it how many blocks it has. This
130 | # let's us know the maximum possible day for which we can request a
131 | # btc price estimate. The time information of blocks is listed in the block
132 | # header, so we ask for the header only when we just need to know the time.
133 | 
134 | 
135 | #get current block height from local node and exit if connection not made
136 | block_count_b = Ask_Node(['getblockcount'])
137 | block_count = int(block_count_b)             #convert text to integer
138 | 
139 | #get block header from current block height
140 | block_hash_b = Ask_Node(['getblockhash',block_count_b])
141 | block_header_b = Ask_Node(['getblockheader',block_hash_b[:64],'true'])
142 | import json #a built in tool for deciphering lists of embedded lists
143 | block_header = json.loads(block_header_b)
144 | 
145 | #get the date and time of the current block height
146 | from datetime import datetime, timezone #built in tools for dates/times
147 | latest_time_in_seconds = block_header['time']
148 | time_datetime = datetime.fromtimestamp(latest_time_in_seconds,tz=timezone.utc)
149 | 
150 | #get the date/time of utc midnight on the latest day
151 | latest_year  = int(time_datetime.strftime("%Y"))
152 | latest_month = int(time_datetime.strftime("%m"))
153 | latest_day   = int(time_datetime.strftime("%d"))
154 | latest_utc_midnight = datetime(latest_year,latest_month,latest_day,0,0,0,tzinfo=timezone.utc)
155 | 
156 | #assign the day before as the latest possible price date
157 | seconds_in_a_day = 60*60*24
158 | yesterday_seconds = latest_time_in_seconds - seconds_in_a_day
159 | latest_price_day = datetime.fromtimestamp(yesterday_seconds,tz=timezone.utc)
160 | latest_price_date = latest_price_day.strftime("%Y-%m-%d")
161 | 
162 | 
163 | # tell the user that a connection has been made and state the lastest price date
164 | print("Connected to local noode at block #:\t"+str(block_count))
165 | print("Latest available price date is: \t"+latest_price_date)
166 | print("Earliest available price date is:\t2020-07-26  (full node)")
167 | 
168 | 
169 | 
170 | 
171 | 
172 | 
173 | 
174 | 
175 | 
176 | 
177 | 
178 | 
179 | 
180 | 
181 | 
182 | ###############################################################################  
183 | 
184 | # Part 3)  Ask for the desired date to estimate the price      
185 | 
186 | ###############################################################################  
187 | 
188 | 
189 | #use python input to get date from the user
190 | date_entered = input("\nEnter date in YYYY-MM-DD (or 'q' to quit):")
191 | 
192 | # quit if desired
193 | if date_entered == 'q':
194 |     exit()
195 | 
196 | #check to see if this is a good date
197 | try:
198 |     year  = int(date_entered.split('-')[0])
199 |     month = int(date_entered.split('-')[1])
200 |     day = int(date_entered.split('-')[2])
201 |     
202 |     #make sure this date is less than the max date
203 |     datetime_entered = datetime(year,month,day,0,0,0,tzinfo=timezone.utc)
204 |     if datetime_entered.timestamp() >= latest_utc_midnight.timestamp():
205 |         print("\nThe date entered is not before the current date, please try again")
206 |         exit()
207 |     
208 |     #make sure this date is after the min date
209 |     july_26_2020 = datetime(2020,7,26,0,0,0,tzinfo=timezone.utc)
210 |     if datetime_entered.timestamp() < july_26_2020.timestamp():
211 |         print("\nThe date entered is before 2020-07-26, please try again")
212 |         exit()
213 | 
214 | except:
215 |     print("\nError interpreting date. Likely not entered in format YYYY-MM-DD")
216 |     print("Please try again\n")
217 |     exit()
218 | 
219 | #get the seconds and printable date string of date entered
220 | price_day_seconds = int(datetime_entered.timestamp())
221 | price_day_date_utc = datetime_entered.strftime("%B %d, %Y")
222 | 
223 | 
224 | 
225 | 
226 | 
227 | 
228 | 
229 | 
230 | 
231 | 
232 | 
233 | 
234 | 
235 | 
236 | 
237 | 
238 | 
239 | ##############################################################################  
240 | 
241 | # Part 4)  Hunt through blocks to find the first block on the target day      
242 | 
243 | ##############################################################################  
244 | 
245 | # This section would be unnecessary if bitcoin Core blocks were organized by time
246 | # instead of by block height. There's no way to ask bitcoin Core for a block at a
247 | # specific time. Instead one must ask for a block, look at it's time, then estimate
248 | # the number of blocks to jump for the next guess. Rinse and repeat.
249 | 
250 | 
251 | #first estimate of the block height of the price day
252 | seconds_since_price_day = latest_time_in_seconds - price_day_seconds
253 | blocks_ago_estimate = round(144*float(seconds_since_price_day)/float(seconds_in_a_day))
254 | price_day_block_estimate = block_count - blocks_ago_estimate
255 | 
256 | #check the time of the price day block estimate
257 | block_hash_b = Ask_Node(['getblockhash',str(price_day_block_estimate)])
258 | block_header_b = Ask_Node(['getblockheader',block_hash_b[:64],'true'])
259 | block_header = json.loads(block_header_b)
260 | time_in_seconds = block_header['time']
261 | 
262 | #get new block estimate from the seconds difference using 144 blocks per day
263 | seconds_difference = time_in_seconds - price_day_seconds
264 | block_jump_estimate = round(144*float(seconds_difference)/float(seconds_in_a_day))
265 | 
266 | #iterate above process until it oscillates around the correct block
267 | last_estimate = 0
268 | last_last_estimate = 0
269 | while block_jump_estimate >6 and block_jump_estimate != last_last_estimate:
270 |     
271 |     #when we osciallate around the correct block, last_last_estimate = block_jump_estimate
272 |     last_last_estimate = last_estimate
273 |     last_estimate = block_jump_estimate
274 |     
275 |     #get block header or new estimate
276 |     price_day_block_estimate = price_day_block_estimate-block_jump_estimate
277 |     block_hash_b = Ask_Node(['getblockhash',str(price_day_block_estimate)])
278 |     block_header_b = Ask_Node(['getblockheader',block_hash_b[:64],'true'])
279 |     block_header = json.loads(block_header_b)
280 |     
281 |     #check time of new block and get new block jump estimate
282 |     time_in_seconds = block_header['time']
283 |     seconds_difference = time_in_seconds - price_day_seconds
284 |     block_jump_estimate = round(144*float(seconds_difference)/float(seconds_in_a_day))
285 |     
286 | #the oscillation may be over multiple blocks so we add/subtract single blocks 
287 | #to ensure we have exactly the first block of the target day
288 | if time_in_seconds > price_day_seconds:
289 |     
290 |     # if the estimate was after price day look at earlier blocks
291 |     while time_in_seconds > price_day_seconds:
292 |         
293 |         #decrement the block by one, read new block header, check time
294 |         price_day_block_estimate = price_day_block_estimate-1
295 |         block_hash_b = Ask_Node(['getblockhash',str(price_day_block_estimate)])
296 |         block_header_b = Ask_Node(['getblockheader',block_hash_b[:64],'true'])
297 |         block_header = json.loads(block_header_b)
298 |         time_in_seconds = block_header['time']
299 |         
300 |     #the guess is now perfectly the first block before midnight
301 |     price_day_block_estimate = price_day_block_estimate + 1
302 | 
303 | # if the estimate was before price day look for later blocks
304 | elif time_in_seconds < price_day_seconds:
305 |     
306 |     while time_in_seconds < price_day_seconds:
307 |         
308 |         #increment the block by one, read new block header, check time
309 |         price_day_block_estimate = price_day_block_estimate+1
310 |         block_hash_b = Ask_Node(['getblockhash',str(price_day_block_estimate)])
311 |         block_header_b = Ask_Node(['getblockheader',block_hash_b[:64],'true'])
312 |         block_header = json.loads(block_header_b)
313 |         time_in_seconds = block_header['time']
314 | 
315 | #assign the estimate as the price day block since it is correct now    
316 | price_day_block = price_day_block_estimate
317 | 
318 | 
319 | 
320 | 
321 | 
322 | 
323 | 
324 | 
325 | 
326 | 
327 | 
328 | 
329 | ##############################################################################
330 | 
331 | #  Part 5) Build the container to hold the output amounts bell curve
332 | 
333 | ##############################################################################
334 | 
335 | # In pure math a bell curve can be perfectly smooth. But to make a bell curve
336 | # from a sample of data, one must specifiy a series of buckets, or bins, and then
337 | # count how many samples are in each bin. If the bin size is too large, say just one
338 | # large bin, a bell curve can't appear because it will have only one bar. The bell 
339 | # curve also doesn't appear if the bin size is too small because then there will 
340 | # only be one sample in each bin and we'd fail to have a distribution of bin heights. 
341 | 
342 | # Although several bin sizes would work, I have found over many years, that 200 bins 
343 | # for every 10x of bitcoin amounts works very well. We use 'every 10x' because just 
344 | # like a long term bitcoin price chart, viewing output amounts in log scale provides 
345 | # a more comprehensive and detailed overview of the amounts being analyzed. 
346 | 
347 | 
348 | # Define the maximum and minimum values (in log10) of btc amounts to use
349 | first_bin_value = -6
350 | last_bin_value = 6  #python -1 means last in list
351 | range_bin_values = last_bin_value - first_bin_value 
352 | 
353 | # create a list of output_bell_curve_bins and add zero sats as the first bin
354 | output_bell_curve_bins = [0.0] #a decimal tells python the list will contain decimals
355 | 
356 | # calculate btc amounts of 200 samples in every 10x from 100 sats (1e-6 btc) to 100k (1e5) btc
357 | for exponent in range(-6,6): #python range uses 'less than' for the big number 
358 |     
359 |     #add 200 bin_width increments in this 10x to the list
360 |     for b in range(0,200):
361 |         
362 |         bin_value = 10 ** (exponent + b/200)
363 |         output_bell_curve_bins.append(bin_value)
364 | 
365 | # Create a list the same size as the bell curve to keep the count of the bins
366 | number_of_bins = len(output_bell_curve_bins)
367 | output_bell_curve_bin_counts = []
368 | for n in range(0,number_of_bins):
369 |     output_bell_curve_bin_counts.append(float(0.0))
370 | 
371 | 
372 | 
373 | 
374 | 
375 | 
376 | 
377 | 
378 | 
379 | 
380 | 
381 | 
382 | 
383 | 
384 | 
385 | 
386 | 
387 | ##############################################################################
388 | 
389 | #  Part 6) Get all output amounts from all block on target day
390 | 
391 | ##############################################################################
392 | 
393 | # This section of the program will take the most time as it requests all 
394 | # all blocks from Core on the price day. It readers every transaction (tx)
395 | # from those blocks and places each tx output value into the bell curve
396 | 
397 | 
398 | 
399 | 
400 | from math import log10 #built in math functions needed logarithms
401 | 
402 | #print header line of update table
403 | print("\nReading all blocks on "+price_day_date_utc+"...")
404 | print("\nThis will take a few minutes (~144 blocks)...")
405 | print("\nHeight\tTime(utc)\t\tTime(32bit)\t\t  Completion %")
406 | 
407 | #get the full data of the first target day block from the node
408 | block_height=price_day_block
409 | block_hash_b = Ask_Node(['getblockhash',str(block_height)])
410 | block_b = Ask_Node(['getblock',block_hash_b[:64],'2'])
411 | block = json.loads(block_b)
412 | 
413 | #get the time of the first block
414 | time_in_seconds = int(block['time'])
415 | time_datetime = datetime.fromtimestamp(time_in_seconds,tz=timezone.utc)
416 | time_utc = time_datetime.strftime("%H:%M:%S")
417 | hour_of_day = int(time_datetime.strftime("%H"))
418 | minute_of_hour = float(time_datetime.strftime("%M"))
419 | day_of_month = int(time_datetime.strftime("%d"))
420 | target_day_of_month = day_of_month
421 | time_32bit = f"{time_in_seconds & 0b11111111111111111111111111111111:32b}"
422 | 
423 | 
424 | #read in blocks until we get a block on the day after the target day
425 | while target_day_of_month == day_of_month:
426 |     
427 |     #get progress estimate
428 |     progress_estimate = 100.0*(hour_of_day+minute_of_hour/60)/24.0
429 |     
430 |     #print progress update
431 |     print(str(block_height)+"\t"+time_utc+"\t"+time_32bit+"\t"+f"{progress_estimate:.2f}"+"%")
432 |     
433 |     #go through all the txs in the block which are stored in a list called 'tx'
434 |     for tx in block['tx']:
435 |         
436 |         #txs have more than one output which are stored in a list called 'vout'
437 |         outputs = tx['vout']
438 |         
439 |         #go through all outputs in the tx
440 |         for output in outputs:
441 |             
442 |             #the bitcoin output amount is called 'value' in Core, add this to the list
443 |             amount = float(output['value'])
444 |             
445 |             #tiny and huge amounts aren't used by the USD price finder
446 |             if 1e-6 < amount < 1e6:
447 |                 
448 |                 #take the log
449 |                 amount_log = log10(amount)
450 |                 
451 |                 #find the right output amount bin to increment
452 |                 percent_in_range = (amount_log-first_bin_value)/range_bin_values
453 |                 bin_number_est = int(percent_in_range * number_of_bins)
454 |                 
455 |                 #search for the exact right bin (won't be less than)
456 |                 while output_bell_curve_bins[bin_number_est] <= amount:
457 |                     bin_number_est += 1
458 |                 bin_number = bin_number_est - 1
459 |                 
460 |                 #increment the output bin
461 |                 output_bell_curve_bin_counts[bin_number] += 1.0   #+= means increment
462 |     
463 |     
464 |     #get the full data of the next block
465 |     block_height = block_height + 1
466 |     block_hash_b = Ask_Node(['getblockhash',str(block_height)])
467 |     block_b = Ask_Node(['getblock',block_hash_b[:64],'2'])
468 |     block = json.loads(block_b)
469 | 
470 |     #get the time of the next block
471 |     time_in_seconds = int(block['time'])
472 |     time_datetime = datetime.fromtimestamp(time_in_seconds,tz=timezone.utc)
473 |     time_utc = time_datetime.strftime("%H:%M:%S")
474 |     day_of_month = int(time_datetime.strftime("%d"))
475 |     minute_of_hour = float(time_datetime.strftime("%M"))
476 |     hour_of_day = int(time_datetime.strftime("%H"))
477 |     time_32bit = f"{time_in_seconds & 0b11111111111111111111111111111111:32b}"
478 | 
479 | 
480 | 
481 | 
482 | 
483 | 
484 | 
485 | 
486 | 
487 | 
488 | 
489 | 
490 | 
491 | 
492 | 
493 | 
494 | ##############################################################################
495 | 
496 | #  Part 7) Remove non-usd related output amounts from the bell curve
497 | 
498 | ##############################################################################
499 | 
500 | 
501 | 
502 | # This sectoins aims to remove non-usd denominated samples from the bell curve
503 | # of outputs. The two primary steps are to remove very large/small outputs
504 | # and then to remove round btc amounts. We don't set the round btc amounts
505 | # to zero because if the USD price of bitcoin is also round, then round
506 | # btc amounts will co-align with round usd amounts. There are many ways to deal
507 | # with this. One way we've found to work is to smooth over the round btc amounts
508 | # using the neighboring amounts in the bell curve. The last step is to normalize
509 | # the bell curve. Normalizing is done by dividing the entire curve by the sum 
510 | # of the curve, and then removing extreme values.
511 | 
512 | 
513 | #remove ouputs below 10k sat (increased from 1k sat in v6)
514 | for n in range(0,401):
515 |     output_bell_curve_bin_counts[n]=0
516 | 
517 | #remove outputs above ten btc
518 | for n in range(1601,number_of_bins):
519 |     output_bell_curve_bin_counts[n]=0
520 | 
521 | #create a list of round btc bin numbers
522 | round_btc_bins = [
523 | 201,  # 1k sats
524 | 401,  # 10k 
525 | 461,  # 20k
526 | 496,  # 30k
527 | 540,  # 50k
528 | 601,  # 100k 
529 | 661,  # 200k
530 | 696,  # 300k
531 | 740,  # 500k
532 | 801,  # 0.01 btc
533 | 861,  # 0.02
534 | 896,  # 0.03
535 | 940,  # 0.04
536 | 1001, # 0.1 
537 | 1061, # 0.2
538 | 1096, # 0.3
539 | 1140, # 0.5
540 | 1201  # 1 btc
541 | ]
542 | 
543 | #smooth over the round btc amounts
544 | for r in round_btc_bins:
545 |     amount_above = output_bell_curve_bin_counts[r+1]
546 |     amount_below = output_bell_curve_bin_counts[r-1]
547 |     output_bell_curve_bin_counts[r] = .5*(amount_above+amount_below)
548 | 
549 | 
550 | #get the sum of the curve
551 | curve_sum = 0.0
552 | for n in range(201,1601):
553 |     curve_sum += output_bell_curve_bin_counts[n]
554 | 
555 | #normalize the curve by dividing by it's sum and removing extreme values
556 | for n in range(201,1601):
557 |     output_bell_curve_bin_counts[n] /= curve_sum
558 |     
559 |     #remove extremes (the iterative process mentioned below found 0.008 to work)
560 |     if output_bell_curve_bin_counts[n] > 0.008:
561 |         output_bell_curve_bin_counts[n] = 0.008
562 |     
563 | 
564 | 
565 | 
566 | 
567 | 
568 | 
569 | 
570 | 
571 | 
572 | 
573 | 
574 | 
575 | 
576 | ##############################################################################
577 | 
578 | #  Part 8) Construct the USD price finder stencil
579 | 
580 | ##############################################################################
581 | 
582 | # We now have a bell curve of outputs which should contain round USD outputs
583 | # as it's prominent features. To expose these prominent features even more,
584 | # and estimate a usd price, we slide a stencil over the bell curve and look 
585 | # for where the slide location is maximized. There are several stencil designs
586 | # and maximization strategies which could accomplish this. The one used here 
587 | # is a stencil whose locations and heights have been found using the averages of
588 | # running this algorithm iteratively over the years 2020-2023. The stencil is
589 | # centered around 0.01 btc = $10,00 as this was the easiest mark to identify.
590 | # The result of this process has produced the following stencil design:
591 | 
592 | 
593 | # create an empty stencil the same size as the bell curve
594 | round_usd_stencil = []
595 | for n in range(0,number_of_bins):
596 |     round_usd_stencil.append(0.0)
597 | 
598 | # fill the round usd stencil with the values found by the process mentioned above
599 | round_usd_stencil[401] = 0.0005957955691168063     # $1
600 | round_usd_stencil[402] = 0.0004454790662303128     # (next one for tx/atm fees)
601 | round_usd_stencil[429] = 0.0001763099393598914     # $1.50
602 | round_usd_stencil[430] = 0.0001851801497144573
603 | round_usd_stencil[461] = 0.0006205616481885794     # $2
604 | round_usd_stencil[462] = 0.0005985696860584984
605 | round_usd_stencil[496] = 0.0006919505728046619     # $3
606 | round_usd_stencil[497] = 0.0008912933078342840
607 | round_usd_stencil[540] = 0.0009372916238804205     # $5
608 | round_usd_stencil[541] = 0.0017125522985034724     # (larger needed range for fees)
609 | round_usd_stencil[600] = 0.0021702347223143030
610 | round_usd_stencil[601] = 0.0037018622326411380     # $10 
611 | round_usd_stencil[602] = 0.0027322168706743802
612 | round_usd_stencil[603] = 0.0016268322583097678     # (larger needed range for fees)
613 | round_usd_stencil[604] = 0.0012601953416497664
614 | round_usd_stencil[661] = 0.0041425242880295460     # $20
615 | round_usd_stencil[662] = 0.0039247767475640830
616 | round_usd_stencil[696] = 0.0032399441632017228     # $30
617 | round_usd_stencil[697] = 0.0037112959007355585
618 | round_usd_stencil[740] = 0.0049921908828370000     # $50
619 | round_usd_stencil[741] = 0.0070636869018197105
620 | round_usd_stencil[801] = 0.0080000000000000000     # $100
621 | round_usd_stencil[802] = 0.0065431388282424440     # (larger needed range for fees)
622 | round_usd_stencil[803] = 0.0044279509203361735
623 | round_usd_stencil[861] = 0.0046132440551747015     # $200
624 | round_usd_stencil[862] = 0.0043647851395531140
625 | round_usd_stencil[896] = 0.0031980892880846567     # $300
626 | round_usd_stencil[897] = 0.0034237641632481910
627 | round_usd_stencil[939] = 0.0025995335505435034     # $500
628 | round_usd_stencil[940] = 0.0032631930982226645     # (larger needed range for fees)
629 | round_usd_stencil[941] = 0.0042753262790881080
630 | round_usd_stencil[1001] =0.0037699501474772350     # $1,000
631 | round_usd_stencil[1002] =0.0030872891064215764     # (larger needed range for fees)
632 | round_usd_stencil[1003] =0.0023237040836798163
633 | round_usd_stencil[1061] =0.0023671764210889895     # $2,000
634 | round_usd_stencil[1062] =0.0020106877104798474
635 | round_usd_stencil[1140] =0.0009099214128654502     # $3,000
636 | round_usd_stencil[1141] =0.0012008546799361498
637 | round_usd_stencil[1201] =0.0007862586076341524     # $10,000
638 | round_usd_stencil[1202] =0.0006900048077192579
639 | 
640 | 
641 | 
642 | 
643 | 
644 | 
645 | 
646 | 
647 | 
648 | 
649 | 
650 | 
651 | 
652 | ##############################################################################
653 | 
654 | #  Part 9) Slide the stencil over the output bell curve to find the best fit
655 | 
656 | ##############################################################################
657 | 
658 | # This is the final step. We slide the stencil over the bell curve and see
659 | # where it fits the best. The best fit location and it's neighbor are used
660 | # in a weighted average to estimate the best fit USD price
661 | 
662 | 
663 | # set up scores for sliding the stencil
664 | best_slide       = 0
665 | best_slide_score = 0.0
666 | total_score      = 0.0
667 | number_of_scores = 0
668 | 
669 | #upper and lower limits for sliding the stencil
670 | min_slide = -200
671 | max_slide = 200
672 | 
673 | #slide the stencil and calculate slide score
674 | for slide in range(min_slide,max_slide):
675 |     
676 |     #shift the bell curve by the slide
677 |     shifted_curve = output_bell_curve_bin_counts[201+slide:1401+slide]
678 |     
679 |     #score the shift by multiplying the curve by the stencil
680 |     slide_score = 0.0
681 |     for n in range(0,len(shifted_curve)):
682 |         slide_score += shifted_curve[n]*round_usd_stencil[n+201]
683 |     
684 |     # increment total and number of scores
685 |     total_score += slide_score
686 |     number_of_scores += 1
687 |     
688 |     # see if this score is the best so far
689 |     if slide_score > best_slide_score:
690 |         best_slide_score = slide_score
691 |         best_slide = slide
692 | 
693 | # estimate the usd price of the best slide
694 | usd100_in_btc_best = output_bell_curve_bins[801+best_slide]
695 | btc_in_usd_best = 100/(usd100_in_btc_best)
696 | 
697 | #find best slide neighbor up
698 | neighbor_up = output_bell_curve_bin_counts[201+best_slide+1:1401+best_slide+1]
699 | neighbor_up_score = 0.0
700 | for n in range(0,len(neighbor_up)):
701 |     neighbor_up_score += neighbor_up[n]*round_usd_stencil[n+201]
702 | 
703 | #find best slide neighbor down
704 | neighbor_down = output_bell_curve_bin_counts[201+best_slide-1:1401+best_slide-1]
705 | neighbor_down_score = 0.0
706 | for n in range(0,len(neighbor_down)):
707 |     neighbor_down_score += neighbor_down[n]*round_usd_stencil[n+201]
708 | 
709 | #get best neighbor
710 | best_neighbor = +1
711 | neighbor_score = neighbor_up_score
712 | if neighbor_down_score > neighbor_up_score:
713 |     best_neighbor = -1
714 |     neighbor_score = neighbor_down_score
715 | 
716 | #get best neighbor usd price
717 | usd100_in_btc_2nd = output_bell_curve_bins[801+best_slide+best_neighbor]
718 | btc_in_usd_2nd = 100/(usd100_in_btc_2nd)
719 | 
720 | #weight average the two usd price estimates
721 | avg_score = total_score/number_of_scores
722 | a1 = best_slide_score - avg_score
723 | a2 = abs(neighbor_score - avg_score)  #theoretically possible to be negative
724 | w1 = a1/(a1+a2)
725 | w2 = a2/(a1+a2)
726 | price_estimate = int(w1*btc_in_usd_best + w2*btc_in_usd_2nd)
727 | 
728 | #report the price estimate
729 | print("\nThe "+price_day_date_utc+" btc price estimate is: $" + f'{price_estimate:,}')
730 | 
731 | 
732 | 
733 |  
734 |     
735 |     
736 |     
737 |     
738 |     
739 |     
740 |     
741 |     
742 |     
743 |     
744 |     


--------------------------------------------------------------------------------
/archive/v8/UTXOracle.py:
--------------------------------------------------------------------------------
  1 | 
  2 | 
  3 | 
  4 | 
  5 | #########################################################################################  
  6 | #                                                                                       #
  7 | #   /$$   /$$ /$$$$$$$$ /$$   /$$  /$$$$$$                               /$$            #
  8 | #  | $$  | $$|__  $$__/| $$  / $$ /$$__  $$                             | $$            #
  9 | #  | $$  | $$   | $$   |  $$/ $$/| $$  \ $$  /$$$$$$  /$$$$$$   /$$$$$$$| $$  /$$$$$$   #
 10 | #  | $$  | $$   | $$    \  $$$$/ | $$  | $$ /$$__  $$|____  $$ /$$_____/| $$ /$$__  $$  #
 11 | #  | $$  | $$   | $$     >$$  $$ | $$  | $$| $$  \__/ /$$$$$$$| $$      | $$| $$$$$$$$  #
 12 | #  | $$  | $$   | $$    /$$/\  $$| $$  | $$| $$      /$$__  $$| $$      | $$| $$_____/  #
 13 | #  |  $$$$$$/   | $$   | $$  \ $$|  $$$$$$/| $$     |  $$$$$$$|  $$$$$$$| $$|  $$$$$$$  #
 14 | #   \______/    |__/   |__/  |__/ \______/ |__/      \_______/ \_______/|__/ \_______/  #
 15 | #                                                                                       #
 16 | #########################################################################################  
 17 | #                     Version 8 - The Smooth Slider                                     
 18 | 
 19 | 
 20 | 
 21 | # UTXOracle is a decentralized alternative to estimating the USD price of bitcoin.
 22 | # Instead of relying on prices given by an exchange, UTXOracle determines the price
 23 | # by analyzing patterns of on-chain transactions. It connects only to a bitcoin
 24 | # node and no other outside sources. It works even with wifi turned off because
 25 | # there are no api or internet communications. Every individual who independently
 26 | # runs this code will produce identical price estimates because even though the algorithm
 27 | # is statistical in nature, both the code and input data are identical.
 28 | # There are no AI or machine learning aspects to this project because black-box style algorithms
 29 | # can create conflicts of interest amongst parties using the price to settle contracts.
 30 | # Every step of the algorithm is fully deterministic, human understandable, and
 31 | # thoroughly documented in the code below.
 32 | 
 33 | # This document is divided following sections:
 34 | 
 35 | # Quick Start) Run it right now
 36 | # Introduction) Background and general description of UTXOracle
 37 | # Part 1) Create a way to talk to your node
 38 | # Part 2) Get the latest block from the node
 39 | # Part 3) Ask the user for a price estimate date (the target date)
 40 | # Part 4) Hunt through blocks to find the first block on the target day
 41 | # Part 5) Build the container to hold the output amounts bell curve
 42 | # Part 6) Get all output amounts from all blocks on the target day
 43 | # Part 7) Remove non-usd related outputs from the bell curve
 44 | # Part 8) Construct the USD price finder stencils
 45 | # Part 9) Estimate the price using the best fit stencil slide 
 46 | 
 47 | 
 48 | 
 49 | ###############################################################################  
 50 | 
 51 | #                        Quick Start          
 52 | 
 53 | ###############################################################################  
 54 | 
 55 | 
 56 | # 1. Make sure you have python3 and bitcoin-cli installed
 57 | # 2. Make sure "server = 1" is in bitcoin.conf
 58 | # 3. Run this file as "python3 UTXOracle.py"
 59 | 
 60 | # If this doesn't work, try filling in your bitcon-cli configuration options:
 61 | 
 62 | 
 63 | # (Optional) configuration options for bitcoin-cli
 64 | datadir       = ""
 65 | rpcuser       = "" 
 66 | rpcpassword   = ""
 67 | rpccookiefile = ""
 68 | rpcconnect    = ""
 69 | rpcport       = ""
 70 | conf          = ""
 71 | 
 72 | 
 73 | 
 74 | 
 75 | 
 76 | 
 77 | 
 78 | ###############################################################################  
 79 | 
 80 | #                   Introduction to the code          
 81 | 
 82 | ###############################################################################  
 83 | 
 84 | 
 85 | # The code that follows is written in a minimalistic style to maximize readability over 
 86 | # efficiency. It is self contained in (this) single file and 
 87 | # defines only one function call so that it can be read top to bottom like reading a paper. 
 88 | # The same code is sometimes repeated instead of defining a function for the 
 89 | # purpose of flow and continuity. There are no dependencies other than standard python 
 90 | # imports because third party libraries improve efficiency at the cost of requiring 
 91 | # the user to download and install third party libraries. 
 92 | 
 93 | # Historical testing of the Oracle shows prices that are accurate within the variance 
 94 | # seen by different bitcoin exchanges themselves. The date and price ranges expected to work 
 95 | # for this version (version 8) are from 2023-12-15 to present and from $5k to $500k. 
 96 | # The previous version of the code (version 7) worked flawlessly from 2020 to 2024 before 
 97 | # ordinal related transactions dramatically changed the character of on-chain output patterns. 
 98 | #
 99 | # If obvious errors are seen in the current version, it will become apparent to everyone 
100 | # (as everyone will get the same result) and a new version will be released. New versions 
101 | # will not be released for slight accuracy improvements as these kinds of releases could be 
102 | # used to manipulate contract settlements using the Oracle.
103 | 
104 | # A basic understanding of how UTXOracle works is as follows:
105 |     
106 | #  Take a day's distribution of bitcoin transaction amounts 
107 | #
108 | #                    * *   *
109 | #                   *   * * *                        
110 | #                  *     *   *   *           
111 | #               * *           * * *           
112 | #          *   * *                  * *      *
113 | #         * * *                      *  * * * *       
114 | #      * *                               * *    * * *
115 | #     * *                                             *
116 | #    *                                      
117 | #   10k sats        0.01 btc           1 btc        10btc 
118 | 
119 | # Create a smooth stencil to align broadly with a typical output day
120 | #
121 | #                       *  *
122 | #                    *         *
123 | #                 *               * 
124 | #              *                     *
125 | #            *                          *
126 | #          *                               *
127 | #        *                                     *
128 | #      *                                            *  
129 | #   10k sats        0.01 btc           1 btc        10btc 
130 | 
131 | # Create a spike stencil that fine tunes the alignment on popular usd amounts
132 | #
133 | #                         *
134 | #                     *   *                       
135 | #                     *   *                   
136 | #                *    *   *          *           
137 | #           *    *    *   *    *     *              
138 | #           *    *    *   *    *     *    *             
139 | #       *   *    *    *   *    *     *    *     *       
140 | #       *   *    *    *   *    *     *    *     *     
141 | #      $1 $10  $20  $50  $100  $500  $1k  $2k   $10k
142 | #
143 | # Slide the smooth and spike stencil over the output data and look for the best fit
144 | 
145 | 
146 | 
147 | 
148 | 
149 | 
150 | 
151 | 
152 | 
153 | 
154 | 
155 | ###############################################################################  
156 | 
157 | #  Part 1) Create a way to talk to your node      
158 | 
159 | ###############################################################################  
160 | 
161 | # We begin by defining a shortcut for calling the node. We do this repeatedly
162 | # throughout the program so it's better to define a function once and call it
163 | # whenever we need it instead of copy and pasting the same code several times. 
164 | # The function asks the node a question and returns the answer to the algorithm 
165 | # where it is needed. If you get an error in this function, the problem is 
166 | # likely that you don't have server=1 in your bitcoin conf file.
167 | 
168 | 
169 | #first we add any node connections options specified by the user above
170 | bitcoin_cli_options = []
171 | if datadir      != "":
172 |     bitcoin_cli_options.append('-datadir='+ datadir)
173 | if rpcuser      != "":
174 |     bitcoin_cli_options.append("-rpcuser="+ rpcuser)
175 | if rpcpassword  != "":
176 |     bitcoin_cli_options.append("-rpcpassword="+ rpcpassword)
177 | if rpccookiefile != "":
178 |     bitcoin_cli_options.append("-rpcookiefile="+ rpccookiefile)
179 | if rpcconnect   != "":
180 |     bitcoin_cli_options.append("-rpcconnect="+ rpcconnect)
181 | if rpcport      != "":
182 |     bitcoin_cli_options.append("-rpcport="+ rpcport)
183 | if conf         != "":
184 |     bitcoin_cli_options.append("-conf="+ conf)
185 | 
186 | 
187 | #import built in python libraries for system commands
188 | import subprocess, sys
189 | 
190 | # now define the function
191 | def Ask_Node(command):
192 |         
193 |     #Here "command" can be any question that your node understands
194 |     
195 |     #We add to the command any configurations options given by the user
196 |     for o in bitcoin_cli_options:
197 |         command.insert(0,o)
198 |     
199 |     #Use Core's default "bitcoin-cli" method of node communication  
200 |     command.insert(0,"bitcoin-cli")
201 |     
202 |     # get the answer from the node and return it to the program
203 |     answer = None
204 |     try:  #python try is used when we need to deal with errors after
205 |         answer = subprocess.check_output(command)
206 |     except Exception as e:
207 |         # something went wrong while getting the answer
208 |         print("Error connecting to your node. Troubleshooting steps:\n")
209 |         print("\t 1) Make sure bitcoin-cli is working. Try command 'bitcoin-cli getblockcount'")
210 |         print("\t 2) Make sure config file bitcoin.conf has server=1")
211 |         print("\t 3) Explore the bitcoin-cli options at the top of UTXOracle.py")
212 |         print("\nThe command was:"+str(command))
213 |         print("\nThe error from bitcoin-cli was:\n")
214 |         print(e)
215 |         sys.exit()
216 |         
217 |     # answer received, return this answer to the program
218 |     return answer
219 | 
220 | 
221 | 
222 | 
223 | 
224 | 
225 | 
226 | 
227 | ###############################################################################  
228 | 
229 | # Part 2)  Get the latest block from the node      
230 | 
231 | ###############################################################################  
232 | 
233 | # The first request to the node is to ask it how many blocks it has. This
234 | # lets us know the maximum possible day for which we can request a
235 | # btc price estimate. The time information of blocks is listed in the block
236 | # header, so we ask for the header only when we just need to know the time.
237 | 
238 | #import built in tools for dates/times and json style lists
239 | from datetime import datetime, timezone, timedelta
240 | import json 
241 | 
242 | #get current block height from local node and exit if connection not made
243 | block_count_b = Ask_Node(['getblockcount'])
244 | block_count = int(block_count_b)             #convert text to integer
245 | 
246 | #get block header from current block height
247 | block_hash_b = Ask_Node(['getblockhash',block_count_b])
248 | block_header_b = Ask_Node(['getblockheader',block_hash_b[:64],'true'])
249 | block_header = json.loads(block_header_b)
250 | 
251 | #get the date and time of the current block height
252 | latest_time_in_seconds = block_header['time']
253 | time_datetime = datetime.fromtimestamp(latest_time_in_seconds,tz=timezone.utc)
254 | 
255 | #get the date/time of utc midnight on the latest day
256 | latest_year  = int(time_datetime.strftime("%Y"))
257 | latest_month = int(time_datetime.strftime("%m"))
258 | latest_day   = int(time_datetime.strftime("%d"))
259 | latest_utc_midnight = datetime(latest_year,latest_month,latest_day,0,0,0,tzinfo=timezone.utc)
260 | 
261 | #assign the day before as the latest possible price date
262 | seconds_in_a_day = 60*60*24
263 | yesterday_seconds = latest_time_in_seconds - seconds_in_a_day
264 | latest_price_day = datetime.fromtimestamp(yesterday_seconds,tz=timezone.utc)
265 | latest_price_date = latest_price_day.strftime("%Y-%m-%d")
266 | 
267 | 
268 | # tell the user that a connection has been made and state the lastest price date
269 | print("UTXOracle version 8")
270 | print("\nConnected to local noode at block #:\t"+str(block_count))
271 | print("Latest available price date:\t\t"+latest_price_date+" (pruned node ok)")
272 | print("Earliest available price date:\t\t2023-12-15 (requires full node)")
273 | 
274 | 
275 | 
276 | 
277 | 
278 | 
279 | 
280 | 
281 | ###############################################################################  
282 | 
283 | # Part 3)  Ask the user for the desired date to estimate the price      
284 | 
285 | ###############################################################################  
286 | 
287 | # In this section we ask the user for a date and make sure that date is in the
288 | # acceptable range for this version.
289 | 
290 | 
291 | date_entered = input("\nEnter date in YYYY-MM-DD format\nor Enter 'q' to quit "+ \
292 |                      "\nor press ENTER for the most recent price:")
293 | 
294 | # quit if desired
295 | if date_entered == 'q':
296 |     sys.exit()
297 |     
298 | # run latest day if hit enter
299 | elif (date_entered == ""):
300 |     datetime_entered = latest_utc_midnight + timedelta(days=-1)
301 |     
302 | #user entered a specific date
303 | else:
304 |     
305 |     #check to see if this is a good date
306 |     try:
307 |         year  = int(date_entered.split('-')[0])
308 |         month = int(date_entered.split('-')[1])
309 |         day = int(date_entered.split('-')[2])
310 |         
311 |         #make sure this date is less than the max date
312 |         datetime_entered = datetime(year,month,day,0,0,0,tzinfo=timezone.utc)
313 |         if datetime_entered.timestamp() > latest_utc_midnight.timestamp():
314 |             print("\nThe date entered is not before the current date, please try again")
315 |             sys.exit()
316 |         
317 |         #make sure this date is after the min date
318 |         dec_15_2023 = datetime(2023,12,15,0,0,0,tzinfo=timezone.utc)
319 |         if datetime_entered.timestamp() < dec_15_2023.timestamp():
320 |             print("\nThe date entered is before 2023-12-15, please try again")
321 |             sys.exit()
322 |     
323 |     except:
324 |         print("\nError interpreting date. Please try again. Make sure format is YYYY-MM-DD")
325 |         sys.exit()
326 | 
327 | 
328 | #get the seconds and printable date string of date entered
329 | price_day_seconds = int(datetime_entered.timestamp())
330 | price_day_date_utc = datetime_entered.strftime("%B %d, %Y")
331 | 
332 | 
333 | print("\n\n########   Starting Price Estimate   ########")
334 | 
335 | 
336 | 
337 | 
338 | 
339 | 
340 | 
341 | ##############################################################################  
342 | 
343 | # Part 4)  Hunt through blocks to find the first block on the target day      
344 | 
345 | ##############################################################################  
346 | 
347 | # Now that we have the target day we need to find which blocks were mined on this day.
348 | # This would be easy if bitcoin Core blocks were organized by time
349 | # instead of by block height. However there's no way to ask bitcoin Core for a block at a
350 | # specific time. Instead one must ask for a block, look at it's time, then estimate
351 | # the number of blocks to jump for the next guess. So we use this
352 | # guess and check method to find all blocks on the target day.
353 | 
354 | 
355 | #first estimate of the block height of the price day
356 | seconds_since_price_day = latest_time_in_seconds - price_day_seconds
357 | blocks_ago_estimate = round(144*float(seconds_since_price_day)/float(seconds_in_a_day))
358 | price_day_block_estimate = block_count - blocks_ago_estimate
359 | 
360 | #check the time of the price day block estimate
361 | block_hash_b = Ask_Node(['getblockhash',str(price_day_block_estimate)])
362 | block_header_b = Ask_Node(['getblockheader',block_hash_b[:64],'true'])
363 | block_header = json.loads(block_header_b)
364 | time_in_seconds = block_header['time']
365 | 
366 | #get new block estimate from the seconds difference using 144 blocks per day
367 | seconds_difference = time_in_seconds - price_day_seconds
368 | block_jump_estimate = round(144*float(seconds_difference)/float(seconds_in_a_day))
369 | 
370 | #iterate above process until it oscillates around the correct block
371 | last_estimate = 0
372 | last_last_estimate = 0
373 | while block_jump_estimate >6 and block_jump_estimate != last_last_estimate:
374 |     
375 |     #when we oscillate around the correct block, last_last_estimate = block_jump_estimate
376 |     last_last_estimate = last_estimate
377 |     last_estimate = block_jump_estimate
378 |     
379 |     #get block header or new estimate
380 |     price_day_block_estimate = price_day_block_estimate-block_jump_estimate
381 |     block_hash_b = Ask_Node(['getblockhash',str(price_day_block_estimate)])
382 |     block_header_b = Ask_Node(['getblockheader',block_hash_b[:64],'true'])
383 |     block_header = json.loads(block_header_b)
384 |     
385 |     #check time of new block and get new block jump estimate
386 |     time_in_seconds = block_header['time']
387 |     seconds_difference = time_in_seconds - price_day_seconds
388 |     block_jump_estimate = round(144*float(seconds_difference)/float(seconds_in_a_day))
389 |     
390 | #the oscillation may be over multiple blocks so we add/subtract single blocks 
391 | #to ensure we have exactly the first block of the target day
392 | if time_in_seconds > price_day_seconds:
393 |     
394 |     # if the estimate was after price day look at earlier blocks
395 |     while time_in_seconds > price_day_seconds:
396 |         
397 |         #decrement the block by one, read new block header, check time
398 |         price_day_block_estimate = price_day_block_estimate-1
399 |         block_hash_b = Ask_Node(['getblockhash',str(price_day_block_estimate)])
400 |         block_header_b = Ask_Node(['getblockheader',block_hash_b[:64],'true'])
401 |         block_header = json.loads(block_header_b)
402 |         time_in_seconds = block_header['time']
403 |         
404 |     #the guess is now perfectly the first block before midnight
405 |     price_day_block_estimate = price_day_block_estimate + 1
406 | 
407 | # if the estimate was before price day look for later blocks
408 | elif time_in_seconds < price_day_seconds:
409 |     
410 |     while time_in_seconds < price_day_seconds:
411 |         
412 |         #increment the block by one, read new block header, check time
413 |         price_day_block_estimate = price_day_block_estimate+1
414 |         block_hash_b = Ask_Node(['getblockhash',str(price_day_block_estimate)])
415 |         block_header_b = Ask_Node(['getblockheader',block_hash_b[:64],'true'])
416 |         block_header = json.loads(block_header_b)
417 |         time_in_seconds = block_header['time']
418 | 
419 | #assign the estimate as the price day block since it is correct now    
420 | price_day_block = price_day_block_estimate
421 | 
422 | 
423 | 
424 | 
425 | 
426 | 
427 | 
428 | 
429 | ##############################################################################
430 | 
431 | #  Part 5) Build the container to hold the output amounts bell curve
432 | 
433 | ##############################################################################
434 | 
435 | # We're almost ready to read in block data but first we must construct the 
436 | # containers which will hold the distribution of transaction output amounts.
437 | # In pure math a bell curve can be perfectly smooth. But to make a bell curve
438 | # from a sample of data, one must specify a series of buckets, or bins, and then
439 | # count how many samples are in each bin. If the bin size is too large, say just one
440 | # large bin, a bell curve can't appear because it will have only one bar. The bell 
441 | # curve also doesn't appear if the bin size is too small because then there will 
442 | # only be one sample in each bin and we'd fail to have a distribution of bin heights. 
443 | # Although several bin sizes would work, I have found over many years, that 200 bins 
444 | # for every 10x of bitcoin amounts works very well. We use 'every 10x' because just 
445 | # like a long term bitcoin price chart, viewing output amounts in log scale provides 
446 | # a more comprehensive and detailed overview of the amounts being analyzed. 
447 | 
448 | 
449 | # Define the maximum and minimum values (in log10) of btc amounts to use
450 | first_bin_value = -6
451 | last_bin_value = 6  #python -1 means last in list
452 | range_bin_values = last_bin_value - first_bin_value 
453 | 
454 | # create a list of output_bell_curve_bins and add zero sats as the first bin
455 | output_bell_curve_bins = [0.0] #a decimal tells python the list will contain decimals
456 | 
457 | # calculate btc amounts of 200 samples in every 10x from 100 sats (1e-6 btc) to 100k (1e5) btc
458 | for exponent in range(-6,6): #python range uses 'less than' for the big number 
459 |     
460 |     #add 200 bin_width increments in this 10x to the list
461 |     for b in range(0,200):
462 |         
463 |         bin_value = 10 ** (exponent + b/200)
464 |         output_bell_curve_bins.append(bin_value)
465 | 
466 | # Create a list the same size as the bell curve to keep the count of the bins
467 | number_of_bins = len(output_bell_curve_bins)
468 | output_bell_curve_bin_counts = []
469 | for n in range(0,number_of_bins):
470 |     output_bell_curve_bin_counts.append(float(0.0))
471 | 
472 | 
473 | 
474 | 
475 | 
476 | 
477 | 
478 | 
479 | 
480 | ##############################################################################
481 | 
482 | #  Part 6) Get all output amounts from all block on target day
483 | 
484 | ##############################################################################
485 | 
486 | # This section of the program will take the most time as it requests all 
487 | # blocks from Core on the price day. It readers every transaction (tx)
488 | # from those blocks and places each tx output value into the bell curve.
489 | # New in version 8 are filters that disallow the following types of transactions
490 | # as they have been found to be unlikely to be round p2p usd transactions: coinbase,
491 | # greater than 5 inputs, greater than 2 outputs, only one output, has op_return,
492 | # has witness data > 500 bytes, and has an input created on the same day.
493 | 
494 | 
495 | 
496 | from math import log10 #built in math functions needed logarithms
497 | 
498 | #print header line of update table
499 | print("\nReading all blocks on "+price_day_date_utc+"...")
500 | print("This will take a few minutes (~144 blocks)...")
501 | print("\nBlock Height\t Block Time(utc)\t\t\tCompletion %")
502 | 
503 | #get the full data of the first target day block from the node
504 | block_height=price_day_block
505 | block_hash_b = Ask_Node(['getblockhash',str(block_height)])
506 | block_b = Ask_Node(['getblock',block_hash_b[:64],'2'])
507 | block = json.loads(block_b)
508 | 
509 | #get the time of the first block
510 | time_in_seconds = int(block['time'])
511 | time_datetime = datetime.fromtimestamp(time_in_seconds,tz=timezone.utc)
512 | time_utc = time_datetime.strftime(" %Y-%m-%d %H:%M:%S")
513 | hour_of_day = int(time_datetime.strftime("%H"))
514 | minute_of_hour = float(time_datetime.strftime("%M"))
515 | day_of_month = int(time_datetime.strftime("%d"))
516 | target_day_of_month = day_of_month
517 | 
518 | # start a list of unique txids using python's "set" variable type
519 | todays_txids = set()
520 | 
521 | 
522 | #read in blocks until we get a block on the day after the target day
523 | while target_day_of_month == day_of_month:
524 |     
525 |     #get progress estimate
526 |     progress_estimate = 100.0*(hour_of_day+minute_of_hour/60)/24.0
527 |     print(str(block_height)+"\t\t"+time_utc+"\t\t"+f"{progress_estimate:.2f}"+"%")
528 |     
529 |     #go through all the txs in the block which are stored in a list called 'tx'
530 |     for tx in block['tx']:
531 |         
532 |         #add txid to todays txids list
533 |         todays_txids.add(tx['txid'][-8:]) #-8 because the tx is unique with only 8 characters
534 |             
535 |         #txs have more than one input which are stored in a list called 'vin'
536 |         inputs = tx['vin']
537 | 
538 |         #txs have more than one output which are stored in a list called 'vout'
539 |         outputs = tx['vout']
540 | 
541 |         #check for coinbase tx
542 |         if "coinbase" in inputs[0]:
543 |             continue  #continue means skip the rest of this transaction
544 | 
545 |         #check for many inputs
546 |         if len(inputs) > 5:
547 |             continue
548 | 
549 |         #check for one outputs
550 |         if len(outputs) < 2:
551 |             continue
552 | 
553 |         #check for many outputs
554 |         if len(outputs) > 2: 
555 |             continue
556 | 
557 |         #check for opreturn
558 |         has_op_return = False
559 |         for output in outputs:
560 |             script_pub_key = output.get("scriptPubKey", {})
561 |             if script_pub_key.get("type") == "nulldata" or "OP_RETURN" in script_pub_key.get("asm", ""):
562 |                 has_op_return = True
563 |                 break
564 |         if has_op_return:
565 |             continue
566 | 
567 |         #go through all inputs s in the tx
568 |         has_sameday_input = False
569 |         has_big_witness = False
570 |         for inpt in inputs:
571 |             
572 |             #look for same day inputs
573 |             if 'txid' in inpt and inpt['txid'][-8:] in todays_txids:
574 |                 has_sameday_input = True
575 |                 break
576 |             
577 |             #look for large witness data (> 500 bytes)
578 |             if "txinwitness" in inpt:
579 |                 for witness in inpt["txinwitness"]:
580 |                     if len(witness) > 500:
581 |                         has_big_witness = True
582 |                         break
583 |             
584 |             #break out of input loop if sameday input of big witness
585 |             if has_sameday_input or has_big_witness:
586 |                 break
587 |             
588 |         #skip the rest of this tx if sameday input of big witness
589 |         if has_sameday_input or has_big_witness:
590 |             continue
591 |             
592 | 
593 |         #go through all outputs in the tx and add the value to the bell curve
594 |         for output in outputs:
595 |             
596 |             #the bitcoin output amount is called 'value' in Core, add this to the list
597 |             amount = float(output['value'])
598 | 
599 |             #tiny and huge amounts aren't used by the USD price finder
600 |             if 1e-5 < amount < 1e5:
601 |                 
602 |                 #take the log
603 |                 amount_log = log10(amount)
604 |                 
605 |                 #find the right output amount bin to increment
606 |                 percent_in_range = (amount_log-first_bin_value)/range_bin_values
607 |                 bin_number_est = int(percent_in_range * number_of_bins)
608 |                 
609 |                 #search for the exact right bin (won't be less than)
610 |                 while output_bell_curve_bins[bin_number_est] <= amount:
611 |                     bin_number_est += 1
612 |                 bin_number = bin_number_est - 1
613 |                 
614 |                 #add this output to the bell curve
615 |                 output_bell_curve_bin_counts[bin_number] += 1.0   #+= means increment
616 |     
617 |     
618 |     #get the full data of the next block
619 |     block_height = block_height + 1
620 |     block_hash_b = Ask_Node(['getblockhash',str(block_height)])
621 |     block_b = Ask_Node(['getblock',block_hash_b[:64],'2'])
622 |     block = json.loads(block_b)
623 | 
624 |     #get the time of the next block
625 |     time_in_seconds = int(block['time'])
626 |     time_datetime = datetime.fromtimestamp(time_in_seconds,tz=timezone.utc)
627 |     time_utc = time_datetime.strftime(" %Y-%m-%d %H:%M:%S")
628 |     day_of_month = int(time_datetime.strftime("%d"))
629 |     minute_of_hour = float(time_datetime.strftime("%M"))
630 |     hour_of_day = int(time_datetime.strftime("%H"))
631 | 
632 | 
633 | 
634 | 
635 | 
636 | 
637 | 
638 | 
639 | ##############################################################################
640 | 
641 | #  Part 7) Remove non-usd related outputs from the bell curve
642 | 
643 | ##############################################################################
644 | 
645 | # This section aims to remove non-usd denominated samples from the bell curve
646 | # of outputs. The two primary steps are to remove very large/small outputs
647 | # and then to remove round btc amounts. We don't set the round btc amounts
648 | # to zero because if the USD price of bitcoin is also round, then round
649 | # btc amounts will co-align with round usd amounts. There are many ways to deal
650 | # with this. One way we've found to work is to smooth over the round btc amounts
651 | # using the neighboring amounts in the bell curve. The last step is to normalize
652 | # the bell curve. Normalizing is done by dividing the entire curve by the sum 
653 | # of the curve. This is done because it's more convenient for signal processing
654 | # procedures if the sum of the signal integrates to one.
655 | 
656 | 
657 | #remove outputs below 10k sat (increased from 1k sat in v6)
658 | for n in range(0,201):
659 |     output_bell_curve_bin_counts[n]=0
660 | 
661 | #remove outputs above ten btc
662 | for n in range(1601,len(output_bell_curve_bin_counts)):
663 |     output_bell_curve_bin_counts[n]=0
664 | 
665 | #create a list of round btc bin numbers
666 | round_btc_bins = [
667 | 201,  # 1k sats
668 | 401,  # 10k 
669 | 461,  # 20k
670 | 496,  # 30k
671 | 540,  # 50k
672 | 601,  # 100k 
673 | 661,  # 200k
674 | 696,  # 300k
675 | 740,  # 500k
676 | 801,  # 0.01 btc
677 | 861,  # 0.02
678 | 896,  # 0.03
679 | 940,  # 0.04
680 | 1001, # 0.1 
681 | 1061, # 0.2
682 | 1096, # 0.3
683 | 1140, # 0.5
684 | 1201  # 1 btc
685 | ]
686 | 
687 | #smooth over the round btc amounts
688 | for r in round_btc_bins:
689 |     amount_above = output_bell_curve_bin_counts[r+1]
690 |     amount_below = output_bell_curve_bin_counts[r-1]
691 |     output_bell_curve_bin_counts[r] = .5*(amount_above+amount_below)
692 | 
693 | 
694 | #get the sum of the curve
695 | curve_sum = 0.0
696 | for n in range(201,1601):
697 |     curve_sum += output_bell_curve_bin_counts[n]
698 | 
699 | #normalize the curve by dividing by it's sum and removing extreme values
700 | for n in range(201,1601):
701 |     output_bell_curve_bin_counts[n] /= curve_sum
702 |     
703 |     #remove extremes (0.008 chosen by historical testing)
704 |     if output_bell_curve_bin_counts[n] > 0.008:
705 |         output_bell_curve_bin_counts[n] = 0.008
706 |     
707 | 
708 | 
709 | 
710 | 
711 | 
712 | 
713 | 
714 | 
715 | ##############################################################################
716 | 
717 | #  Part 8) Construct the USD price finder stencils
718 | 
719 | ##############################################################################
720 | 
721 | # We now have a bell curve of outputs which should contain round USD outputs
722 | # as it's prominent features. To expose these prominent features more,
723 | # and estimate a usd price, we slide two types of stencils over the bell curve and look 
724 | # for where the slide location is maximized. There are several stencil designs
725 | # and maximization strategies which could accomplish this. The one used here 
726 | # is to have one smooth stencil that finds the general shape of a typical output
727 | # distribution day, and a spike stencil which narrows in on exact locations
728 | # of the round USD amounts. Both the smooth and spike stenciled have been created
729 | # by an iterative process of manually sliding together round USD spikes in
730 | # output distribtutions over every day from 2020 to 2024, and then taking the average
731 | # general shape and round usd spike values over that period.
732 | 
733 | # Load the average smooth stencil to align broadly with a typical output day 
734 | #
735 | #                       *  *
736 | #                    *         *
737 | #                 *               * 
738 | #              *                     *
739 | #            *                          *
740 | #          *                               *
741 | #        *                                     *
742 | #      *                                            *  
743 | #   10k sats        0.01 btc           1 btc        10btc 
744 | 
745 | # Parameters
746 | num_elements = 803
747 | mean = 411 #(num_elements - 1) / 2  # Center of the array
748 | std_dev = 201
749 | 
750 | smooth_stencil = []
751 | for x in range(num_elements):
752 |     exp_part = -((x - mean) ** 2) / (2 * (std_dev ** 2))
753 |     smooth_stencil.append( (.00150 * 2.718281828459045 ** exp_part) + (.0000005 * x) )
754 | 
755 | 
756 | # Load the average spike stencil that fine tunes the alignment on popular usd amounts
757 | #
758 | #                         *
759 | #                     *   *                       
760 | #                     *   *                   
761 | #                *    *   *          *           
762 | #           *    *    *   *    *     *              
763 | #           *    *    *   *    *     *    *             
764 | #       *   *    *    *   *    *     *    *     *       
765 | #       *   *    *    *   *    *     *    *     *     
766 | #      $1 $10  $20  $50  $100  $500  $1k  $2k   $10k
767 | 
768 | spike_stencil = []
769 | for n in range(0,803):
770 |     spike_stencil.append(0.0)
771 |     
772 | #round usd bin location   #popularity    #usd amount  
773 | spike_stencil[40] = 0.001300198324984352  # $1
774 | spike_stencil[141]= 0.001676746949820743  # $5
775 | spike_stencil[201]= 0.003468805546942046  # $10
776 | spike_stencil[202]= 0.001991977522512513  # 
777 | spike_stencil[236]= 0.001905066647961839  # $15
778 | spike_stencil[261]= 0.003341772718156079  # $20
779 | spike_stencil[262]= 0.002588902624584287  # 
780 | spike_stencil[296]= 0.002577893841190244  # $30
781 | spike_stencil[297]= 0.002733728814200412  # 
782 | spike_stencil[340]= 0.003076117748975647  # $50
783 | spike_stencil[341]= 0.005613067550103145  # 
784 | spike_stencil[342]= 0.003088253178535568  # 
785 | spike_stencil[400]= 0.002918457489366139  # $100
786 | spike_stencil[401]= 0.006174500465286022  # 
787 | spike_stencil[402]= 0.004417068070043504  # 
788 | spike_stencil[403]= 0.002628663628020371  # 
789 | spike_stencil[436]= 0.002858828161543839  # $150
790 | spike_stencil[461]= 0.004097463611984264  # $200
791 | spike_stencil[462]= 0.003345917406120509  # 
792 | spike_stencil[496]= 0.002521467726855856  # $300
793 | spike_stencil[497]= 0.002784125730361008  # 
794 | spike_stencil[541]= 0.003792850444811335  # $500
795 | spike_stencil[601]= 0.003688240815848247  # $1000
796 | spike_stencil[602]= 0.002392400117402263  # 
797 | spike_stencil[636]= 0.001280993059008106  # $1500
798 | spike_stencil[661]= 0.001654665137536031  # $2000
799 | spike_stencil[662]= 0.001395501347054946  # 
800 | spike_stencil[741]= 0.001154279140906312  # $5000
801 | spike_stencil[801]= 0.000832244504868709  # $10000
802 | 
803 | 
804 | 
805 | 
806 | 
807 | 
808 | 
809 | ##############################################################################
810 | 
811 | #  Part 9) Part 9) Estimate the price using the best fit stencil slide
812 | 
813 | ##############################################################################
814 | 
815 | # This is the concluding step. We slide the stencil over the bell curve and see
816 | # where it fits the best. The best fit location and it's neighbor are used
817 | # in a weighted average to estimate the best fit USD price
818 | 
819 | 
820 | # set up scores for sliding the stencil
821 | best_slide        = 0
822 | best_slide_score  = 0
823 | total_score       = 0
824 | 
825 | #weighting of the smooth and spike slide scores
826 | smooth_weight     = 0.65
827 | spike_weight      = 1
828 | 
829 | #establish the center slide such that if zero slide then 0.001 btc is $100 ($100k price)
830 | center_p001 = 601   # 601 is where 0.001 btc is in the output bell curve
831 | left_p001   = center_p001 - int((len(spike_stencil) +1)/2)
832 | right_p001  = center_p001 + int((len(spike_stencil) +1)/2)
833 | 
834 | #upper and lower limits for sliding the stencil
835 | min_slide = -141   # $500k
836 | max_slide =  201   # $5k
837 |     
838 | #slide the stencil and calculate slide score
839 | for slide in range(min_slide,max_slide):
840 |     
841 |     #shift the bell curve by the slide
842 |     shifted_curve = output_bell_curve_bin_counts[left_p001+slide:right_p001+slide]
843 |     
844 |     #score the smoothslide by multiplying the curve by the stencil
845 |     slide_score_smooth = 0.0
846 |     for n in range(0,len(smooth_stencil)):
847 |         slide_score_smooth += shifted_curve[n]*smooth_stencil[n]
848 |     
849 |     #score the spiky slide by multiplying the curve by the stencil
850 |     slide_score = 0.0
851 |     for n in range(0,len(spike_stencil)):
852 |         slide_score += shifted_curve[n]*spike_stencil[n]
853 |     
854 |     # add the spike and smooth slide scores, neglect smooth slide over wrong regions
855 |     if slide < 150:
856 |         slide_score = slide_score + slide_score_smooth*.65
857 |         
858 |     # see if this score is the best so far
859 |     if slide_score > best_slide_score:
860 |         best_slide_score = slide_score
861 |         best_slide = slide
862 |     
863 |     # increment the total score
864 |     total_score += slide_score
865 |         
866 | # estimate the usd price of the best slide
867 | usd100_in_btc_best = output_bell_curve_bins[center_p001+best_slide]
868 | btc_in_usd_best = 100/(usd100_in_btc_best)
869 | 
870 | #find best slide neighbor up
871 | neighbor_up = output_bell_curve_bin_counts[left_p001+best_slide+1:right_p001+best_slide+1]
872 | neighbor_up_score = 0.0
873 | for n in range(0,len(spike_stencil)):
874 |     neighbor_up_score += neighbor_up[n]*spike_stencil[n]
875 | 
876 | #find best slide neighbor down
877 | neighbor_down = output_bell_curve_bin_counts[left_p001+best_slide-1:right_p001+best_slide-1]
878 | neighbor_down_score = 0.0
879 | for n in range(0,len(spike_stencil)):
880 |     neighbor_down_score += neighbor_down[n]*spike_stencil[n]
881 | 
882 | #get best neighbor
883 | best_neighbor = +1
884 | neighbor_score = neighbor_up_score
885 | if neighbor_down_score > neighbor_up_score:
886 |     best_neighbor = -1
887 |     neighbor_score = neighbor_down_score
888 | 
889 | #get best neighbor usd price
890 | usd100_in_btc_2nd = output_bell_curve_bins[center_p001+best_slide+best_neighbor]
891 | btc_in_usd_2nd = 100/(usd100_in_btc_2nd)
892 | 
893 | #weight average the two usd price estimates
894 | avg_score = total_score/len(range(min_slide,max_slide))
895 | a1 = best_slide_score - avg_score
896 | a2 = abs(neighbor_score - avg_score)
897 | w1 = a1/(a1+a2)
898 | w2 = a2/(a1+a2)
899 | price_estimate = int(w1*btc_in_usd_best + w2*btc_in_usd_2nd)
900 | 
901 | # Finally, report the price estimate
902 | print("\nThe "+price_day_date_utc+" btc price estimate is: $" + f'{price_estimate:,}')
903 | 
904 | 
905 | 
906 | 
907 | 
908 | 
909 | 
910 | ##############################################################################
911 | 
912 | #  License
913 | 
914 | ##############################################################################
915 | 
916 | # This software is free to use, modify and distribute for non-financial gain purposes,
917 | # so long as the full license is included with any use or redistribution.
918 | 
919 | # Any use of this software for financial gain, including but not limited to
920 | # commercial applications, paid services, or monetized redistribution, requires
921 | # the expressed written consent of the author (@SteveSimple on x.com).
922 | 
923 | 
924 | 


--------------------------------------------------------------------------------
/archive/start9/start9/UTXOracle.py:
--------------------------------------------------------------------------------
   1 | 
   2 | #########################################################################################  
   3 | #                                                                                       #
   4 | #   /$$   /$$ /$$$$$$$$ /$$   /$$  /$$$$$$                               /$$            #
   5 | #  | $$  | $$|__  $$__/| $$  / $$ /$$__  $$                             | $$            #
   6 | #  | $$  | $$   | $$   |  $$/ $$/| $$  \ $$  /$$$$$$  /$$$$$$   /$$$$$$$| $$  /$$$$$$   #
   7 | #  | $$  | $$   | $$    \  $$$$/ | $$  | $$ /$$__  $$|____  $$ /$$_____/| $$ /$$__  $$  #
   8 | #  | $$  | $$   | $$     >$$  $$ | $$  | $$| $$  \__/ /$$$$$$$| $$      | $$| $$$$$$$$  #
   9 | #  | $$  | $$   | $$    /$$/\  $$| $$  | $$| $$      /$$__  $$| $$      | $$| $$_____/  #
  10 | #  |  $$$$$$/   | $$   | $$  \ $$|  $$$$$$/| $$     |  $$$$$$$|  $$$$$$$| $$|  $$$$$$$  #
  11 | #   \______/    |__/   |__/  |__/ \______/ |__/      \_______/ \_______/|__/ \_______/  #
  12 | #                                                                                       #
  13 | #########################################################################################  
  14 | #                     Start 9 implementation                                     
  15 | 
  16 | 
  17 | # set platform dependent data paths and clear terminal
  18 | import platform
  19 | import os
  20 | data_dir = []
  21 | system = platform.system()
  22 | if system == "Darwin":  # macOS
  23 |     data_dir = os.path.expanduser("~/Library/Application Support/Bitcoin")
  24 |     #os.system('clear')
  25 | elif system == "Windows":
  26 |     data_dir = os.path.join(os.environ.get("APPDATA", ""), "Bitcoin")
  27 |     #os.system('cls')
  28 | else:  # Linux or others
  29 |     data_dir = os.path.expanduser("~/.bitcoin")
  30 |     #os.system('clear')
  31 | 
  32 | # initialize variables for blocks and dates
  33 | date_entered = ""
  34 | date_mode = True
  35 | block_mode = False
  36 | block_start_num = 0
  37 | block_finish_num = 0
  38 | block_nums_needed = []
  39 | block_hashes_needed = []
  40 | block_times_needed = []
  41 | 
  42 | #print help text for the user if needed
  43 | import sys
  44 | def print_help():
  45 |     help_text = """
  46 | Usage: python script.py [options]
  47 | 
  48 | Options:
  49 |   -h               Show this help message
  50 |   -d YYYY/MM/DD    Specify a UTC date to evaluate
  51 |   -p /path/to/dir  Specify the data directory for blk files
  52 |   -rb              Use last 144 recent blocks instead of date mode
  53 | """
  54 |     print(help_text)
  55 |     sys.exit(0)
  56 | 
  57 | #did use ask for help
  58 | if "-h" in sys.argv:
  59 |     print_help()
  60 |     
  61 | #did user specify a date?
  62 | if "-d" in sys.argv:
  63 |     h_index = sys.argv.index("-d")
  64 |     if h_index + 1 < len(sys.argv):
  65 |         date_entered = sys.argv[h_index + 1]
  66 | 
  67 | #did user specify a data path?
  68 | if "-p" in sys.argv:
  69 |     d_index = sys.argv.index("-p")
  70 |     if d_index + 1 < len(sys.argv):
  71 |         data_dir = sys.argv[d_index + 1]
  72 | 
  73 | #did user specify blocks instead of a date?
  74 | if "-rb" in sys.argv:
  75 |     date_mode = False
  76 |     block_mode = True
  77 | 
  78 | # Look for bitcoin.conf or bitcoin_rw.conf
  79 | conf_path = None
  80 | conf_candidates = ["bitcoin.conf", "bitcoin_rw.conf"]
  81 | for fname in conf_candidates:
  82 |     path = os.path.join(data_dir, fname)
  83 |     if os.path.exists(path):
  84 |         conf_path = path
  85 |         break
  86 | if not conf_path:
  87 |     print(f"Invalid Bitcoin data directory: {data_dir}")
  88 |     print("Expected to find 'bitcoin.conf' or 'bitcoin_rw.conf' in this directory.")
  89 |     sys.exit(1)
  90 | 
  91 | # Parse the conf file
  92 | conf_settings = {}
  93 | with open(conf_path, 'r') as f:
  94 |     for line in f:
  95 |         line = line.strip()
  96 |         if not line or line.startswith("#"):
  97 |             continue
  98 |         if "=" in line:
  99 |             key, value = line.split("=", 1)
 100 |             conf_settings[key.strip()] = value.strip().strip('"')
 101 | 
 102 | # Set blocks directory
 103 | blocks_dir = os.path.expanduser(conf_settings.get("blocksdir", os.path.join(data_dir, "blocks")))
 104 | 
 105 | # Prepare RPC parameters
 106 | rpc_user = conf_settings.get("rpcuser")
 107 | rpc_password = conf_settings.get("rpcpassword")
 108 | cookie_path = conf_settings.get("rpccookiefile", os.path.join(data_dir, ".cookie"))
 109 | rpc_host = conf_settings.get("rpcconnect", "127.0.0.1")
 110 | rpc_port = int(conf_settings.get("rpcport", "8332"))
 111 | 
 112 | # Prepare bitcoin-cli fallback options (if needed elsewhere)
 113 | bitcoin_cli_options = []
 114 | if rpc_user and rpc_password:
 115 |     bitcoin_cli_options.append(f"-rpcuser={rpc_user}")
 116 |     bitcoin_cli_options.append(f"-rpcpassword={rpc_password}")
 117 | else:
 118 |     if os.path.exists(cookie_path):
 119 |         bitcoin_cli_options.append(f"-rpccookiefile={cookie_path}")
 120 | 
 121 | if rpc_host:
 122 |     bitcoin_cli_options.append(f"-rpcconnect={rpc_host}")
 123 | if "rpcport" in conf_settings:
 124 |     bitcoin_cli_options.append(f"-rpcport={conf_settings['rpcport']}")
 125 | 
 126 | 
 127 | 
 128 | ###############################################################################  
 129 | 
 130 | #  Part 1) Create a way to talk to your node      
 131 | 
 132 | ###############################################################################  
 133 | 
 134 | # Here we define a shortcut for calling the node. We do this repeatedly
 135 | # throughout the program so it's better to define a function once and call it
 136 | # whenever we need it instead of copy and pasting the same code several times. 
 137 | # The function asks the node a question and returns the answer to the algorithm 
 138 | # where it is needed. If you get an error in this function, the problem is 
 139 | # likely that you don't have server=1 in your bitcoin conf file.
 140 | 
 141 | print("\nCurrent operation  \t\t\t\tTotal Completion",flush=True)
 142 | print("\nConnecting to node...",flush=True)
 143 | print("0%..", end="",flush=True)
 144 | 
 145 | # define the node communication function
 146 | import http.client
 147 | import json
 148 | import base64
 149 | 
 150 | def Ask_Node(command, cred_creation):
 151 |     method = command[0]
 152 |     params = command[1:]
 153 | 
 154 |     # Handle authentication
 155 |     rpc_u = rpc_user
 156 |     rpc_p = rpc_password
 157 | 
 158 |     if not rpc_u or not rpc_p:
 159 |         try:
 160 |             with open(cookie_path, "r") as f:
 161 |                 cookie = f.read().strip()
 162 |                 rpc_u, rpc_p = cookie.split(":", 1)
 163 |         except Exception as e:
 164 |             print("Error reading .cookie file for RPC authentication.")
 165 |             print("Details:", e)
 166 |             sys.exit(1)
 167 | 
 168 |     # Prepare JSON-RPC payload
 169 |     payload = json.dumps({
 170 |         "jsonrpc": "1.0",
 171 |         "id": "utxoracle",
 172 |         "method": method,
 173 |         "params": params
 174 |     })
 175 | 
 176 |     # Basic auth header
 177 |     auth_header = base64.b64encode(f"{rpc_u}:{rpc_p}".encode()).decode()
 178 |     headers = {
 179 |         "Content-Type": "application/json",
 180 |         "Authorization": f"Basic {auth_header}"
 181 |     }
 182 | 
 183 |     try:
 184 |         conn = http.client.HTTPConnection(rpc_host, rpc_port)
 185 |         conn.request("POST", "/", payload, headers)
 186 |         response = conn.getresponse()
 187 |         if response.status != 200:
 188 |             raise Exception(f"HTTP error {response.status} {response.reason}")
 189 |         raw_data = response.read()
 190 |         conn.close()
 191 | 
 192 |         # Extract result and mimic `subprocess.check_output` return (as bytes)
 193 |         parsed = json.loads(raw_data)
 194 |         if parsed.get("error"):
 195 |             raise Exception(parsed["error"])
 196 |         result = parsed["result"]
 197 |         if isinstance(result, (dict, list)):
 198 |             return json.dumps(result, indent=2).encode()  # pretty-print like CLI
 199 |         else:
 200 |             return str(result).encode()
 201 | 
 202 |     except Exception as e:
 203 |         if not cred_creation:
 204 |             print("Error connecting to your node via RPC. Troubleshooting steps:\n")
 205 |             print("\t1) Ensure bitcoind or bitcoin-qt is running with server=1.")
 206 |             print("\t2) Check rpcuser/rpcpassword or .cookie.")
 207 |             print("\t3) Verify RPC port/host settings.")
 208 |             print("\nThe attempted RPC method was:", method)
 209 |             print("Parameters:", params)
 210 |             print("\nThe error was:\n", e)
 211 |             sys.exit(1)
 212 | 
 213 | 
 214 | 
 215 | 
 216 | 
 217 | 
 218 | ###############################################################################  
 219 | 
 220 | # Part 2)  Get the latest block from the node      
 221 | 
 222 | ###############################################################################  
 223 | 
 224 | # The first request to the node is to ask it how many blocks it has. This
 225 | # lets us know the maximum possible day for which we can request a
 226 | # btc price estimate. The time information of blocks is listed in the block
 227 | # header, so we ask for the header only when we just need to know the time.
 228 | 
 229 | #import built in tools for dates/times and json style lists
 230 | from datetime import datetime, timezone, timedelta
 231 | 
 232 | #get current block height from local node and exit if connection not made
 233 | print("20%..", end="",flush=True)
 234 | Ask_Node(['getblockcount'], True) #create RPC creds if necessary
 235 | block_count_b = Ask_Node(['getblockcount'], False)
 236 | print("40%..", end="",flush=True)
 237 | block_count = int(block_count_b)             #convert text to integer
 238 | block_count_consensus = block_count-6
 239 | 
 240 | #get block header from current block height
 241 | #block_hash = Ask_Node(['getblockhash', str(block_count_consensus)]).decode().strip()
 242 | block_hash = Ask_Node(['getblockhash',block_count_consensus], False).decode().strip()
 243 | print("60%..", end="",flush=True)
 244 | block_header_b = Ask_Node(['getblockheader', block_hash, True], False)
 245 | block_header = json.loads(block_header_b)
 246 | print("80%..", end="",flush=True)
 247 | 
 248 | 
 249 | #get the date and time of the current block height
 250 | latest_time_in_seconds = block_header['time']
 251 | time_datetime = datetime.fromtimestamp(latest_time_in_seconds,tz=timezone.utc)
 252 | 
 253 | #get the date/time of utc midnight on the latest day
 254 | latest_year  = int(time_datetime.strftime("%Y"))
 255 | latest_month = int(time_datetime.strftime("%m"))
 256 | latest_day   = int(time_datetime.strftime("%d"))
 257 | latest_utc_midnight = datetime(latest_year,latest_month,latest_day,0,0,0,tzinfo=timezone.utc)
 258 | 
 259 | #assign the day before as the latest possible price date
 260 | seconds_in_a_day = 60*60*24
 261 | yesterday_seconds = latest_time_in_seconds - seconds_in_a_day
 262 | latest_price_day = datetime.fromtimestamp(yesterday_seconds,tz=timezone.utc)
 263 | latest_price_date = latest_price_day.strftime("%Y-%m-%d")
 264 | print("100%", end="",flush=True)
 265 | 
 266 | #print completion update
 267 | print("\t\t\t5% done",flush=True)
 268 | 
 269 | 
 270 | 
 271 | ###############################################################################  
 272 | 
 273 | # Part 3) Check that the date entered is an acceptable date    
 274 | 
 275 | ###############################################################################  
 276 | 
 277 | # In this section make sure that the date requested is in the
 278 | # acceptable range for this version. This section is not used if user 
 279 | # specificed block numbers instead of a date
 280 | 
 281 | if date_mode:
 282 | 
 283 |     # run latest day if hit enter
 284 |     if (date_entered == ""):
 285 |         datetime_entered = latest_utc_midnight + timedelta(days=-1)
 286 |         
 287 |     #user entered a specific date
 288 |     else:
 289 |         
 290 |         #check to see if this is a good date
 291 |         try:
 292 |             year  = int(date_entered.split('/')[0])
 293 |             month = int(date_entered.split('/')[1])
 294 |             day = int(date_entered.split('/')[2])
 295 |             
 296 |             #make sure this date is less than the max date
 297 |             datetime_entered = datetime(year,month,day,0,0,0,tzinfo=timezone.utc)
 298 |             if datetime_entered.timestamp() >= latest_utc_midnight.timestamp():
 299 |                 print("\nDate is after the latest avaiable. We need 6 blocks after UTC midnight.")
 300 |                 print("Run UTXOracle.py -rb for the most recent blocks")
 301 |                 
 302 |                 sys.exit()
 303 |             
 304 |             #make sure this date is after the min date
 305 |             dec_15_2023 = datetime(2023,12,15,0,0,0,tzinfo=timezone.utc)
 306 |             if datetime_entered.timestamp() < dec_15_2023.timestamp():
 307 |                 print("\nThe date entered is before 2023-12-15, please try again")
 308 |                 sys.exit()
 309 |         
 310 |         except:
 311 |             print("\nError interpreting date. Please try again. Make sure format is YYYY/MM/DD")
 312 |             sys.exit()
 313 |     
 314 |     
 315 |     #get the seconds and printable date string of date entered
 316 |     price_day_seconds = int(datetime_entered.timestamp())
 317 |     price_day_date_utc = datetime_entered.strftime("%b %d, %Y")
 318 |     price_date_dash = datetime_entered.strftime("%Y-%m-%d")
 319 |     
 320 | 
 321 | 
 322 | 
 323 | 
 324 | ##############################################################################  
 325 | 
 326 | # Part 4) Find the all blocks on the target day or in block height range requested
 327 | 
 328 | ##############################################################################  
 329 | 
 330 | # Now that we have the target day we need to find which blocks were mined on this day.
 331 | # This would be easy if bitcoin blocks were organized by time
 332 | # instead of by block height. However there's no way to ask bitcoin for a block at a
 333 | # specific time. Instead one must ask for a block, look at it's time, then estimate
 334 | # the number of blocks to jump for the next guess. So we use this
 335 | # guess and check method to find all blocks on the target day.
 336 | 
 337 | #define a shortcut for getting the block time from the block number
 338 | def get_block_time(height):
 339 |     block_hash_b = Ask_Node(['getblockhash',height], False)
 340 |     #block_header_b = Ask_Node(['getblockheader',block_hash_b[:64],True])
 341 |     block_header_b = Ask_Node(['getblockheader', block_hash_b.decode().strip(), True], False)
 342 | 
 343 |     block_header = json.loads(block_header_b)
 344 |     return(block_header['time'], block_hash_b[:64].decode())
 345 | 
 346 | #define a shortcut for getting the day of money from a time in seconds
 347 | def get_day_of_month(time_in_seconds):
 348 |     time_datetime = datetime.fromtimestamp(time_in_seconds,tz=timezone.utc)
 349 |     return(int(time_datetime.strftime("%d")))
 350 | 
 351 | #if block mode add the blocks and hashes to a list
 352 | if block_mode:
 353 |     
 354 |     print("\nFinding the last 144 blocks",flush=True)
 355 |     
 356 |     #get the last block number of the day
 357 |     block_finish_num = block_count
 358 |     block_start_num = block_finish_num - 144
 359 |     
 360 |     #append needed block nums and hashes needed
 361 |     block_num = block_start_num
 362 |     time_in_seconds, hash_end = get_block_time(block_start_num)
 363 |     print_every = 0
 364 |     while block_num < block_finish_num:
 365 |         
 366 |         #print update
 367 |         if (block_num-block_start_num)/144*100 > print_every and print_every < 100:
 368 |             print(str(print_every)+"%..",end="",flush=True)
 369 |             print_every += 20
 370 |         block_nums_needed.append(block_num)
 371 |         block_hashes_needed.append(hash_end)
 372 |         block_times_needed.append(time_in_seconds)
 373 |         block_num += 1
 374 |         time_in_seconds, hash_end = get_block_time(block_num)
 375 |         
 376 |     print("100%\t\t\t25% done",flush=True)
 377 | 
 378 | #if date mode search for all the blocks on this day
 379 | elif date_mode:
 380 |     
 381 |     print("\nFinding first blocks on "+datetime_entered.strftime("%b %d, %Y"),flush=True)
 382 |     print("0%..",end="", flush=True)
 383 |     #first estimate of the block height of the price day
 384 |     seconds_since_price_day = latest_time_in_seconds - price_day_seconds
 385 |     blocks_ago_estimate = round(144*float(seconds_since_price_day)/float(seconds_in_a_day))
 386 |     price_day_block_estimate = block_count_consensus - blocks_ago_estimate
 387 |     
 388 |     #check the time of the price day block estimate
 389 |     time_in_seconds, hash_end = get_block_time(price_day_block_estimate) 
 390 |     
 391 |     #get new block estimate from the seconds difference using 144 blocks per day
 392 |     print("20%..",end="",flush=True)
 393 |     seconds_difference = time_in_seconds - price_day_seconds
 394 |     block_jump_estimate = round(144*float(seconds_difference)/float(seconds_in_a_day))
 395 |     
 396 |     #iterate above process until it oscillates around the correct block
 397 |     last_estimate = 0
 398 |     last_last_estimate = 0
 399 |     
 400 |     print("40%..",end="",flush=True)
 401 |     while block_jump_estimate >6 and block_jump_estimate != last_last_estimate:
 402 |         
 403 |         #when we oscillate around the correct block, last_last_estimate = block_jump_estimate
 404 |         last_last_estimate = last_estimate
 405 |         last_estimate = block_jump_estimate
 406 |         
 407 |         #get block header or new estimate
 408 |         price_day_block_estimate = price_day_block_estimate-block_jump_estimate
 409 |         
 410 |         #check time of new block and get new block jump estimate
 411 |         time_in_seconds, hash_end = get_block_time(price_day_block_estimate) 
 412 |         seconds_difference = time_in_seconds - price_day_seconds
 413 |         block_jump_estimate = round(144*float(seconds_difference)/float(seconds_in_a_day))
 414 |     
 415 |     print("60%..",end="",flush=True)
 416 |     #the oscillation may be over multiple blocks so we add/subtract single blocks 
 417 |     #to ensure we have exactly the first block of the target day
 418 |     if time_in_seconds > price_day_seconds:
 419 |         
 420 |         # if the estimate was after price day look at earlier blocks
 421 |         while time_in_seconds > price_day_seconds:
 422 |             
 423 |             #decrement the block by one, read new block header, check time
 424 |             price_day_block_estimate = price_day_block_estimate-1
 425 |             time_in_seconds, hash_end = get_block_time(price_day_block_estimate) 
 426 |             
 427 |         #the guess is now perfectly the first block before midnight
 428 |         price_day_block_estimate = price_day_block_estimate + 1
 429 |         
 430 |     # if the estimate was before price day look for later blocks
 431 |     elif time_in_seconds < price_day_seconds:
 432 |         
 433 |         while time_in_seconds < price_day_seconds:
 434 |             
 435 |             #increment the block by one, read new block header, check time
 436 |             price_day_block_estimate = price_day_block_estimate+1
 437 |             time_in_seconds, hash_end = get_block_time(price_day_block_estimate) 
 438 |     
 439 |     print("80%..",end="",flush=True)
 440 |     #assign the estimate as the price day block since it is correct now    
 441 |     price_day_block = price_day_block_estimate
 442 |     
 443 |     #get the day of the month 
 444 |     time_in_seconds, hash_start = get_block_time(price_day_block)
 445 |     day1 = get_day_of_month(time_in_seconds)
 446 |     
 447 |     #get the last block number of the day
 448 |     price_day_block_end = price_day_block 
 449 |     time_in_seconds, hash_end = get_block_time(price_day_block_end)
 450 |     day2 = get_day_of_month(time_in_seconds)
 451 |     
 452 |     print("100%\t\t\t25% done",flush=True)
 453 |     print("\nFinding last blocks on "+datetime_entered.strftime("%b %d, %Y"),flush=True)
 454 |     
 455 |     #load block nums and hashes needed
 456 |     block_num = 0
 457 |     print_next = 0
 458 |     while day1 == day2:
 459 |         
 460 |         #print progress update
 461 |         block_num+=1
 462 |         if block_num/144 * 100 > print_next:
 463 |             if print_next < 100:
 464 |                 print(str(print_next)+"%..",end="",flush=True)
 465 |                 print_next +=20
 466 |         
 467 |         #append needed block
 468 |         block_nums_needed.append(price_day_block_end)
 469 |         block_hashes_needed.append(hash_end)
 470 |         block_times_needed.append(time_in_seconds)
 471 |         price_day_block_end += 1 #assume 30+ blocks this day
 472 |         time_in_seconds, hash_end = get_block_time(price_day_block_end)
 473 |         day2 = get_day_of_month(time_in_seconds)
 474 |     
 475 |     #complete print update status
 476 |     while print_next<100:
 477 |         print(str(print_next)+"%..",end="",flush=True)
 478 |         print_next +=20
 479 |     
 480 |     #set start and end block numbers
 481 |     block_start_num = price_day_block
 482 |     block_finish_num = price_day_block_end
 483 | 
 484 |     print("100%\t\t\t50% done",flush=True)
 485 | 
 486 | 
 487 | 
 488 | 
 489 | 
 490 | 
 491 | ##############################################################################
 492 | 
 493 | #  Part 5) Build the container to hold the output amounts bell curve
 494 | 
 495 | ##############################################################################
 496 | 
 497 | # We're almost ready to read in block data but first we must construct the 
 498 | # containers which will hold the distribution of transaction output amounts.
 499 | # In pure math a bell curve can be perfectly smooth. But to make a bell curve
 500 | # from a sample of data, one must specify a series of buckets, or bins, and then
 501 | # count how many samples are in each bin. If the bin size is too large, say just one
 502 | # large bin, a bell curve can't appear because it will have only one bar. The bell 
 503 | # curve also doesn't appear if the bin size is too small because then there will 
 504 | # only be one sample in each bin and we'd fail to have a distribution of bin heights. 
 505 | # Although several bin sizes would work, I have found over many years, that 200 bins 
 506 | # for every 10x of bitcoin amounts works very well. We use 'every 10x' because just 
 507 | # like a long term bitcoin price chart, viewing output amounts in log scale provides 
 508 | # a more comprehensive and detailed overview of the amounts being analyzed. 
 509 | 
 510 | # Define the maximum and minimum values (in log10) of btc amounts to use
 511 | first_bin_value = -6
 512 | last_bin_value = 6  #python -1 means last in list
 513 | range_bin_values = last_bin_value - first_bin_value 
 514 | 
 515 | # create a list of output_bell_curve_bins and add zero sats as the first bin
 516 | output_bell_curve_bins = [0.0] #a decimal tells python the list will contain decimals
 517 | 
 518 | # calculate btc amounts of 200 samples in every 10x from 100 sats (1e-6 btc) to 100k (1e5) btc
 519 | for exponent in range(-6,6): #python range uses 'less than' for the big number 
 520 |     
 521 |     #add 200 bin_width increments in this 10x to the list
 522 |     for b in range(0,200):
 523 |         
 524 |         bin_value = 10 ** (exponent + b/200)
 525 |         output_bell_curve_bins.append(bin_value)
 526 | 
 527 | # Create a list the same size as the bell curve to keep the count of the bins
 528 | number_of_bins = len(output_bell_curve_bins)
 529 | output_bell_curve_bin_counts = []
 530 | for n in range(0,number_of_bins):
 531 |     output_bell_curve_bin_counts.append(float(0.0))
 532 | 
 533 | 
 534 | 
 535 | 
 536 | 
 537 | ##############################################################################
 538 | 
 539 | #  Part 6) Get all output amounts from all block on target day
 540 | 
 541 | ##############################################################################
 542 | 
 543 | # This section of the program will take the most time as it requests all 
 544 | # blocks from the node on the price day. It readers every transaction (tx)
 545 | # from those blocks and places each tx output value into the bell curve.
 546 | # New in version 8 are filters that disallow the following types of transactions
 547 | # as they have been found to be unlikely to be round p2p usd transactions: coinbase,
 548 | # greater than 5 inputs, greater than 2 outputs, only one output, has op_return,
 549 | # has witness data > 500 bytes, and has an input created on the same day.
 550 | 
 551 | print("\nLoading every transaction from every block",flush=True)
 552 | 
 553 | 
 554 | 
 555 | # #initialize output lists and variables
 556 | from struct import unpack
 557 | import binascii
 558 | todays_txids = set()
 559 | raw_outputs = []
 560 | block_heights_dec = []
 561 | block_times_dec = []
 562 | print_next = 0
 563 | block_num = 0
 564 | 
 565 | 
 566 | #shortcut for reading bytes of data from the block file
 567 | import struct
 568 | from math import log10 
 569 | import hashlib
 570 | def read_varint(f):
 571 |     i = f.read(1)
 572 |     if not i:
 573 |         return 0
 574 |     i = i[0]
 575 |     if i < 0xfd:
 576 |         return i
 577 |     elif i == 0xfd:
 578 |         val = struct.unpack("<H", f.read(2))[0]
 579 |     elif i == 0xfe:
 580 |         val = struct.unpack("<I", f.read(4))[0]
 581 |     else:
 582 |         val = struct.unpack("<Q", f.read(8))[0]
 583 |     return val
 584 | 
 585 | #shortcut for encoding variable size integers to bytes
 586 | from io import BytesIO
 587 | def encode_varint(i: int) -> bytes:
 588 |     assert i >= 0
 589 |     if i < 0xfd:
 590 |         return i.to_bytes(1, 'little')
 591 |     elif i <= 0xffff:
 592 |         return b'\xfd' + i.to_bytes(2, 'little')
 593 |     elif i <= 0xffffffff:
 594 |         return b'\xfe' + i.to_bytes(4, 'little')
 595 |     else:
 596 |         return b'\xff' + i.to_bytes(8, 'little')
 597 | 
 598 | #shortcut for computing the txid because blk files don't store txids
 599 | def compute_txid(raw_tx_bytes: bytes) -> bytes:
 600 |     
 601 |     stream = BytesIO(raw_tx_bytes)
 602 | 
 603 |     # Read version
 604 |     version = stream.read(4)
 605 | 
 606 |     # Peek at marker/flag to detect SegWit
 607 |     marker = stream.read(1)
 608 |     flag = stream.read(1)
 609 |     is_segwit = (marker == b'\x00' and flag == b'\x01')
 610 | 
 611 |     if not is_segwit:
 612 |         # Legacy tx: rewind and hash full raw tx
 613 |         stream.seek(0)
 614 |         stripped_tx = stream.read()
 615 |     else:
 616 |         # Start stripped tx with version
 617 |         stripped_tx = bytearray()
 618 |         stripped_tx += version
 619 | 
 620 |         # Inputs
 621 |         input_count = read_varint(stream)
 622 |         stripped_tx += encode_varint(input_count)
 623 |         for _ in range(input_count):
 624 |             stripped_tx += stream.read(32)  # prev txid
 625 |             stripped_tx += stream.read(4)   # vout index
 626 |             script_len = read_varint(stream)
 627 |             stripped_tx += encode_varint(script_len)
 628 |             stripped_tx += stream.read(script_len)
 629 |             stripped_tx += stream.read(4)   # sequence
 630 | 
 631 |         # Outputs
 632 |         output_count = read_varint(stream)
 633 |         stripped_tx += encode_varint(output_count)
 634 |         for _ in range(output_count):
 635 |             stripped_tx += stream.read(8)   # value
 636 |             script_len = read_varint(stream)
 637 |             stripped_tx += encode_varint(script_len)
 638 |             stripped_tx += stream.read(script_len)
 639 | 
 640 |         # Skip witness data
 641 |         for _ in range(input_count):
 642 |             stack_count = read_varint(stream)
 643 |             for _ in range(stack_count):
 644 |                 item_len = read_varint(stream)
 645 |                 stream.read(item_len)
 646 | 
 647 |         # Locktime
 648 |         stripped_tx += stream.read(4)
 649 | 
 650 |     return hashlib.sha256(hashlib.sha256(stripped_tx).digest()).digest()[::-1]
 651 | 
 652 | 
 653 | # read in all blocks needed
 654 | for bh in block_hashes_needed:
 655 |     block_num += 1
 656 |     print_progress = block_num / len(block_hashes_needed) * 100
 657 |     if print_progress > print_next and print_next < 100:
 658 |         #print(f"{int(print_next)}% ", end="", flush=True)
 659 |         print(f"{int(print_next)}%..",end="",flush=True)
 660 |         print_next += 1
 661 |         if print_next % 7 == 0:
 662 |             print("\n", end="")
 663 | 
 664 |     # Get raw block hex using RPC
 665 |     raw_block_hex = Ask_Node(["getblock", bh, 0], False).decode().strip()
 666 |     raw_block_bytes = binascii.unhexlify(raw_block_hex)
 667 |     stream = BytesIO(raw_block_bytes)
 668 | 
 669 |     # Read header (skip 80 bytes)
 670 |     stream.read(80)
 671 |     tx_count = read_varint(stream)
 672 |     
 673 |     # loop through all txs in this block
 674 |     txs_to_add = []
 675 |     for tx_index in range(tx_count):
 676 |         start_tx = stream.tell()
 677 |         version = stream.read(4)
 678 | 
 679 |         # Check for SegWit
 680 |         marker_flag = stream.read(2)
 681 |         is_segwit = marker_flag == b'\x00\x01'
 682 |         if not is_segwit:
 683 |             stream.seek(start_tx + 4)
 684 | 
 685 |         # Read inputs
 686 |         input_count = read_varint(stream)
 687 |         inputs = []
 688 |         has_op_return = False
 689 |         witness_exceeds = False
 690 |         is_coinbase = False
 691 |         input_txids = []
 692 |         for _ in range(input_count):
 693 |             prev_txid = stream.read(32)
 694 |             prev_index = stream.read(4)
 695 |             script_len = read_varint(stream)
 696 |             script = stream.read(script_len)
 697 |             stream.read(4)
 698 |             input_txids.append(prev_txid[::-1].hex())
 699 |             if prev_txid == b'\x00' * 32 and prev_index == b'\xff\xff\xff\xff':
 700 |                 is_coinbase = True
 701 |             inputs.append({"script": script})
 702 | 
 703 |         #read outputs
 704 |         output_count = read_varint(stream)
 705 |         output_values = []
 706 |         for _ in range(output_count):
 707 |             value_sats = unpack("<Q", stream.read(8))[0]
 708 |             script_len = read_varint(stream)
 709 |             script = stream.read(script_len)
 710 |             if script and script[0] == 0x6a:
 711 |                 has_op_return = True
 712 |             value_btc = value_sats / 1e8
 713 |             if 1e-5 < value_btc < 1e5:
 714 |                 output_values.append(value_btc)
 715 | 
 716 |         # check witness data
 717 |         if is_segwit:
 718 |             for input_data in inputs:
 719 |                 stack_count = read_varint(stream)
 720 |                 total_witness_len = 0
 721 |                 for _ in range(stack_count):
 722 |                     item_len = read_varint(stream)
 723 |                     total_witness_len += item_len
 724 |                     stream.read(item_len)
 725 |                     if item_len > 500 or total_witness_len > 500:
 726 |                         witness_exceeds = True
 727 | 
 728 |         #comput txid
 729 |         stream.read(4)
 730 |         end_tx = stream.tell()
 731 |         stream.seek(start_tx)
 732 |         raw_tx = stream.read(end_tx - start_tx)
 733 |         txid = compute_txid(raw_tx)
 734 |         todays_txids.add(txid.hex())
 735 | 
 736 |         #check same day tx
 737 |         is_same_day_tx = any(itxid in todays_txids for itxid in input_txids)
 738 | 
 739 |         # apply filter and add output to bell curve
 740 |         if (input_count <= 5 and output_count == 2 and not is_coinbase and
 741 |             not has_op_return and not witness_exceeds and not is_same_day_tx):
 742 |             for amount in output_values:
 743 |                 amount_log = log10(amount)
 744 |                 percent_in_range = (amount_log - first_bin_value) / range_bin_values
 745 |                 bin_number_est = int(percent_in_range * number_of_bins)
 746 |                 while output_bell_curve_bins[bin_number_est] <= amount:
 747 |                     bin_number_est += 1
 748 |                 bin_number = bin_number_est - 1
 749 |                 output_bell_curve_bin_counts[bin_number] += 1.0
 750 |                 txs_to_add.append(amount)
 751 | 
 752 |     # add outputs to raw outputs
 753 |     if len(txs_to_add) > 0:
 754 |         bkh = block_nums_needed[block_num - 1]
 755 |         tm = block_times_needed[block_num - 1]
 756 |         for amt in txs_to_add:
 757 |             raw_outputs.append(amt)
 758 |             block_heights_dec.append(bkh)
 759 |             block_times_dec.append(tm)
 760 | 
 761 | 
 762 | print("100% \t\t\t\t\t\t95% done",flush=True)
 763 | 
 764 |             
 765 | 
 766 | ##############################################################################
 767 | 
 768 | #  Part 9) Remove non-usd related outputs from the bell curve
 769 | 
 770 | ##############################################################################
 771 | 
 772 | # This section aims to remove non-usd denominated samples from the bell curve
 773 | # of outputs. The two primary steps are to remove very large/small outputs
 774 | # and then to remove round btc amounts. We don't set the round btc amounts
 775 | # to zero because if the USD price of bitcoin is also round, then round
 776 | # btc amounts will co-align with round usd amounts. There are many ways to deal
 777 | # with this. One way we've found to work is to smooth over the round btc amounts
 778 | # using the neighboring amounts in the bell curve. The last step is to normalize
 779 | # the bell curve. Normalizing is done by dividing the entire curve by the sum 
 780 | # of the curve. This is done because it's more convenient for signal processing
 781 | # procedures if the sum of the signal integrates to one.
 782 | 
 783 | # print update
 784 | print("\nFinding prices and rendering plot",flush=True)
 785 | print("0%..",end="",flush=True)
 786 | 
 787 | #remove outputs below 10k sat (increased from 1k sat in v6)
 788 | for n in range(0,201):
 789 |     output_bell_curve_bin_counts[n]=0
 790 | 
 791 | #remove outputs above ten btc
 792 | for n in range(1601,len(output_bell_curve_bin_counts)):
 793 |     output_bell_curve_bin_counts[n]=0
 794 | 
 795 | #create a list of round btc bin numbers
 796 | round_btc_bins = [
 797 | 201,  # 1k sats
 798 | 401,  # 10k 
 799 | 461,  # 20k
 800 | 496,  # 30k
 801 | 540,  # 50k
 802 | 601,  # 100k 
 803 | 661,  # 200k
 804 | 696,  # 300k
 805 | 740,  # 500k
 806 | 801,  # 0.01 btc
 807 | 861,  # 0.02
 808 | 896,  # 0.03
 809 | 940,  # 0.04
 810 | 1001, # 0.1 
 811 | 1061, # 0.2
 812 | 1096, # 0.3
 813 | 1140, # 0.5
 814 | 1201  # 1 btc
 815 | ]
 816 | 
 817 | #smooth over the round btc amounts
 818 | for r in round_btc_bins:
 819 |     amount_above = output_bell_curve_bin_counts[r+1]
 820 |     amount_below = output_bell_curve_bin_counts[r-1]
 821 |     output_bell_curve_bin_counts[r] = .5*(amount_above+amount_below)
 822 | 
 823 | #get the sum of the curve
 824 | curve_sum = 0.0
 825 | for n in range(201,1601):
 826 |     curve_sum += output_bell_curve_bin_counts[n]
 827 | 
 828 | #normalize the curve by dividing by it's sum and removing extreme values
 829 | for n in range(201,1601):
 830 |     output_bell_curve_bin_counts[n] /= curve_sum
 831 |     
 832 |     #remove extremes (0.008 chosen by historical testing)
 833 |     if output_bell_curve_bin_counts[n] > 0.008:
 834 |         output_bell_curve_bin_counts[n] = 0.008
 835 | 
 836 | #print update    
 837 | print("20%..",end="",flush=True)
 838 | 
 839 | 
 840 | 
 841 | 
 842 | ##############################################################################
 843 | 
 844 | #  Part 7) Construct the USD price finder stencils
 845 | 
 846 | ##############################################################################
 847 | 
 848 | # We now have a bell curve of outputs which should contain round USD outputs
 849 | # as it's prominent features. To expose these prominent features more,
 850 | # and estimate a usd price, we slide two types of stencils over the bell curve and look 
 851 | # for where the slide location is maximized. There are several stencil designs
 852 | # and maximization strategies which could accomplish this. The one used here 
 853 | # is to have one smooth stencil that finds the general shape of a typical output
 854 | # distribution day, and a spike stencil which narrows in on exact locations
 855 | # of the round USD amounts. Both the smooth and spike stenciled have been created
 856 | # by an iterative process of manually sliding together round USD spikes in
 857 | # output distribtutions over every day from 2020 to 2024, and then taking the average
 858 | # general shape and round usd spike values over that period.
 859 | 
 860 | # Load the average smooth stencil to align broadly with a typical output day 
 861 | #
 862 | #                       *  *
 863 | #                    *         *
 864 | #                 *               * 
 865 | #              *                     *
 866 | #            *                          *
 867 | #          *                               *
 868 | #        *                                     *
 869 | #      *                                            *  
 870 | #   10k sats        0.01 btc           1 btc        10btc 
 871 | 
 872 | # Parameters
 873 | num_elements = 803
 874 | mean = 411 #(num_elements - 1) / 2  # Center of the array
 875 | std_dev = 201
 876 | 
 877 | #contstruct the smooth stencil
 878 | smooth_stencil = []
 879 | for x in range(num_elements):
 880 |     exp_part = -((x - mean) ** 2) / (2 * (std_dev ** 2))
 881 |     smooth_stencil.append( (.00150 * 2.718281828459045 ** exp_part) + (.0000005 * x) )
 882 | 
 883 | # Load the spike stencil that fine tunes the alignment on popular usd amounts
 884 | #
 885 | #                         *
 886 | #                     *   *                       
 887 | #                     *   *                   
 888 | #                *    *   *          *           
 889 | #           *    *    *   *    *     *              
 890 | #           *    *    *   *    *     *    *             
 891 | #       *   *    *    *   *    *     *    *     *       
 892 | #       *   *    *    *   *    *     *    *     *     
 893 | #      $1 $10  $20  $50  $100  $500  $1k  $2k   $10k
 894 | 
 895 | spike_stencil = []
 896 | for n in range(0,803):
 897 |     spike_stencil.append(0.0)
 898 |     
 899 | #round usd bin location   #popularity    #usd amount  
 900 | spike_stencil[40] = 0.001300198324984352  # $1
 901 | spike_stencil[141]= 0.001676746949820743  # $5
 902 | spike_stencil[201]= 0.003468805546942046  # $10
 903 | spike_stencil[202]= 0.001991977522512513  # 
 904 | spike_stencil[236]= 0.001905066647961839  # $15
 905 | spike_stencil[261]= 0.003341772718156079  # $20
 906 | spike_stencil[262]= 0.002588902624584287  # 
 907 | spike_stencil[296]= 0.002577893841190244  # $30
 908 | spike_stencil[297]= 0.002733728814200412  # 
 909 | spike_stencil[340]= 0.003076117748975647  # $50
 910 | spike_stencil[341]= 0.005613067550103145  # 
 911 | spike_stencil[342]= 0.003088253178535568  # 
 912 | spike_stencil[400]= 0.002918457489366139  # $100
 913 | spike_stencil[401]= 0.006174500465286022  # 
 914 | spike_stencil[402]= 0.004417068070043504  # 
 915 | spike_stencil[403]= 0.002628663628020371  # 
 916 | spike_stencil[436]= 0.002858828161543839  # $150
 917 | spike_stencil[461]= 0.004097463611984264  # $200
 918 | spike_stencil[462]= 0.003345917406120509  # 
 919 | spike_stencil[496]= 0.002521467726855856  # $300
 920 | spike_stencil[497]= 0.002784125730361008  # 
 921 | spike_stencil[541]= 0.003792850444811335  # $500
 922 | spike_stencil[601]= 0.003688240815848247  # $1000
 923 | spike_stencil[602]= 0.002392400117402263  # 
 924 | spike_stencil[636]= 0.001280993059008106  # $1500
 925 | spike_stencil[661]= 0.001654665137536031  # $2000
 926 | spike_stencil[662]= 0.001395501347054946  # 
 927 | spike_stencil[741]= 0.001154279140906312  # $5000
 928 | spike_stencil[801]= 0.000832244504868709  # $10000
 929 | 
 930 | 
 931 | 
 932 | ##############################################################################
 933 | 
 934 | #  Part 8) Estimate a rough price using the best fit stencil slide
 935 | 
 936 | ##############################################################################
 937 | 
 938 | # Here we slide the stencil over the bell curve and see
 939 | # where it fits the best. The best fit location and it's neighbor are used
 940 | # in a weighted average to estimate the best fit USD price
 941 | 
 942 | # set up scores for sliding the stencil
 943 | best_slide        = 0
 944 | best_slide_score  = 0
 945 | total_score       = 0
 946 | 
 947 | #weighting of the smooth and spike slide scores
 948 | smooth_weight     = 0.65
 949 | spike_weight      = 1
 950 | 
 951 | #establish the center slide such that if zero slide then 0.001 btc is $100 ($100k price)
 952 | center_p001 = 601   # 601 is where 0.001 btc is in the output bell curve
 953 | left_p001   = center_p001 - int((len(spike_stencil) +1)/2)
 954 | right_p001  = center_p001 + int((len(spike_stencil) +1)/2)
 955 | 
 956 | #upper and lower limits for sliding the stencil
 957 | min_slide = -141   # $500k
 958 | max_slide =  201   # $5k
 959 |     
 960 | #slide the stencil and calculate slide score
 961 | for slide in range(min_slide,max_slide):
 962 |     
 963 |     #shift the bell curve by the slide
 964 |     shifted_curve = output_bell_curve_bin_counts[left_p001+slide:right_p001+slide]
 965 |     
 966 |     #score the smoothslide by multiplying the curve by the stencil
 967 |     slide_score_smooth = 0.0
 968 |     for n in range(0,len(smooth_stencil)):
 969 |         slide_score_smooth += shifted_curve[n]*smooth_stencil[n]
 970 |     
 971 |     #score the spiky slide by multiplying the curve by the stencil
 972 |     slide_score = 0.0
 973 |     for n in range(0,len(spike_stencil)):
 974 |         slide_score += shifted_curve[n]*spike_stencil[n]
 975 |     
 976 |     # add the spike and smooth slide scores, neglect smooth slide over wrong regions
 977 |     if slide < 150:
 978 |         slide_score = slide_score + slide_score_smooth*.65
 979 |         
 980 |     # see if this score is the best so far
 981 |     if slide_score > best_slide_score:
 982 |         best_slide_score = slide_score
 983 |         best_slide = slide
 984 |     
 985 |     # increment the total score
 986 |     total_score += slide_score
 987 |         
 988 | # estimate the usd price of the best slide
 989 | usd100_in_btc_best = output_bell_curve_bins[center_p001+best_slide]
 990 | btc_in_usd_best = 100/(usd100_in_btc_best)
 991 | 
 992 | #find best slide neighbor up
 993 | neighbor_up = output_bell_curve_bin_counts[left_p001+best_slide+1:right_p001+best_slide+1]
 994 | neighbor_up_score = 0.0
 995 | for n in range(0,len(spike_stencil)):
 996 |     neighbor_up_score += neighbor_up[n]*spike_stencil[n]
 997 | 
 998 | #find best slide neighbor down
 999 | neighbor_down = output_bell_curve_bin_counts[left_p001+best_slide-1:right_p001+best_slide-1]
1000 | neighbor_down_score = 0.0
1001 | for n in range(0,len(spike_stencil)):
1002 |     neighbor_down_score += neighbor_down[n]*spike_stencil[n]
1003 | 
1004 | #get best neighbor
1005 | best_neighbor = +1
1006 | neighbor_score = neighbor_up_score
1007 | if neighbor_down_score > neighbor_up_score:
1008 |     best_neighbor = -1
1009 |     neighbor_score = neighbor_down_score
1010 | 
1011 | #get best neighbor usd price
1012 | usd100_in_btc_2nd = output_bell_curve_bins[center_p001+best_slide+best_neighbor]
1013 | btc_in_usd_2nd = 100/(usd100_in_btc_2nd)
1014 | 
1015 | #weight average the two usd price estimates
1016 | avg_score = total_score/len(range(min_slide,max_slide))
1017 | a1 = best_slide_score - avg_score
1018 | a2 = abs(neighbor_score - avg_score)
1019 | w1 = a1/(a1+a2)
1020 | w2 = a2/(a1+a2)
1021 | rough_price_estimate = int(w1*btc_in_usd_best + w2*btc_in_usd_2nd)
1022 | 
1023 | # Print update
1024 | print("40%..",end="",flush=True)
1025 | 
1026 | 
1027 | 
1028 | 
1029 | ##############################################################################
1030 | 
1031 | #  Part 9) Convert all outputs near round USD to the USD price used in the output
1032 | 
1033 | ##############################################################################
1034 | 
1035 | # In this section we converting the outputs to the price used by those outputs to
1036 | # create a round USD amount. we also further remove micro round sat amounts (new in v 9)
1037 | 
1038 | 
1039 | # list of round USD prices to collect outputs from
1040 | usds = [5,10,15,20,25,30,40,50,100,150,200,300,500,1000]
1041 | 
1042 | # pct of price increase of decrease to include
1043 | pct_range_wide = .25
1044 | 
1045 | # filter for micro round satoshi amounts
1046 | micro_remove_list = []
1047 | i = .00005000
1048 | while i<.0001:
1049 |     micro_remove_list.append(i)
1050 |     i += .00001
1051 | i = .0001
1052 | while i<.001:
1053 |     micro_remove_list.append(i)
1054 |     i += .00001
1055 | i = .001
1056 | while i<.01:
1057 |     micro_remove_list.append(i)
1058 |     i += .0001
1059 | i = .01
1060 | while i<.1:
1061 |     micro_remove_list.append(i)
1062 |     i += .001
1063 | i = .1
1064 | while i<1:
1065 |     micro_remove_list.append(i)
1066 |     i += .01
1067 | pct_micro_remove = .0001
1068 | 
1069 | # init output prices list
1070 | output_prices = []
1071 | output_blocks = []
1072 | output_times = []
1073 | 
1074 | #loop through all outputs
1075 | for i in range (0,len(raw_outputs)):
1076 |     
1077 |     #get the amount, height and time of the next output
1078 |     n = raw_outputs[i]
1079 |     b = block_heights_dec[i]
1080 |     t = block_times_dec[i]
1081 |     
1082 |     #loop throughll usd amounts possible
1083 |     for usd in usds:
1084 |         
1085 |         #calculate the upper and lower bounds for the USD range
1086 |         avbtc = usd/rough_price_estimate
1087 |         btc_up = avbtc + pct_range_wide * avbtc 
1088 |         btc_dn = avbtc - pct_range_wide * avbtc
1089 |     
1090 |         # check if inside price bounds
1091 |         if btc_dn < n < btc_up:
1092 |             append = True
1093 |             
1094 |             #remove perfectly round sats
1095 |             for r in micro_remove_list:
1096 |                 
1097 |                 rm_dn = r - pct_micro_remove * r
1098 |                 rm_up = r + pct_micro_remove * r
1099 |                 if rm_dn < n < rm_up:
1100 |                     append = False
1101 |             
1102 |             # if in price range and not perfectly round sat, add to list
1103 |             if append:
1104 |                 output_prices.append(usd/n)
1105 |                 output_blocks.append(b)
1106 |                 output_times.append(t)
1107 | 
1108 | print("60%..",end="",flush=True)
1109 | 
1110 | 
1111 | 
1112 | 
1113 | ##############################################################################
1114 | 
1115 | #  Part 10) Find the central output and average deviation for plot window
1116 | 
1117 | ##############################################################################
1118 | 
1119 | # Here we use and iterative procedure of finding a central output,
1120 | # shifting the price ranges to re-center the data, then finding the center again.
1121 | # This asserts that the price is the center of the largest cluster of price
1122 | # points of the day. This has been found by testing to be better than
1123 | # the mean or the median because we specifically need the cluster center
1124 | 
1125 | # define an algorithm for finding the central price point and avg deviation
1126 | def find_central_output(r2, price_min, price_max):
1127 |     
1128 |     #sort the list of prices
1129 |     r6 = [r for r in r2 if price_min < r < price_max]
1130 |     outputs = sorted(r6)
1131 |     n = len(outputs)
1132 | 
1133 |     # Prefix sums
1134 |     prefix_sum = []
1135 |     total = 0
1136 |     for x in outputs:
1137 |         total += x
1138 |         prefix_sum.append(total)
1139 | 
1140 |     # count the number of point left and right
1141 |     left_counts = list(range(n))
1142 |     right_counts = [n - i - 1 for i in left_counts]
1143 |     left_sums = [0] + prefix_sum[:-1]
1144 |     right_sums = [total - x for x in prefix_sum]
1145 | 
1146 |     # find the total distance to other points
1147 |     total_dists = []
1148 |     for i in range(n):
1149 |         dist = (outputs[i] * left_counts[i] - left_sums[i]) + (right_sums[i] - outputs[i] * right_counts[i])
1150 |         total_dists.append(dist)
1151 | 
1152 |     # find the Most central output
1153 |     min_index, _ = min(enumerate(total_dists), key=lambda x: x[1])
1154 |     best_output = outputs[min_index]
1155 | 
1156 |     # Median absolute deviation
1157 |     deviations = [abs(x - best_output) for x in outputs]
1158 |     deviations.sort()
1159 |     m = len(deviations)
1160 |     if m % 2 == 0:
1161 |         mad = (deviations[m//2 - 1] + deviations[m//2]) / 2
1162 |     else:
1163 |         mad = deviations[m//2]
1164 | 
1165 |     return best_output, mad
1166 | 
1167 | 
1168 | # use a tight pct range to find the first central price
1169 | pct_range_tight = .05
1170 | price_up = rough_price_estimate + pct_range_tight * rough_price_estimate 
1171 | price_dn = rough_price_estimate - pct_range_tight * rough_price_estimate
1172 | central_price, av_dev = find_central_output(output_prices,price_dn,price_up)
1173 | 
1174 | # find the deviation as a percentage of the price range
1175 | price_range = price_up - price_dn
1176 | dev_pct = av_dev/price_range
1177 | 
1178 | # iteratively re-center the bounds and find a new center price until convergence
1179 | avs = set()
1180 | avs.add(central_price)
1181 | while central_price not in avs:
1182 |     avs.add(central_price)
1183 |     price_up = central_price + pct_range_tight * central_price 
1184 |     price_dn = central_price - pct_range_tight * central_price
1185 |     central_price, av_dev = find_central_output(output_prices,price_dn,price_up)
1186 |     price_range = price_up - price_dn
1187 |     dev_pct = av_dev/price_range
1188 | 
1189 | #print update
1190 | print("80%..",end="",flush=True)
1191 | 
1192 | #because price flutucation may exceed bounds, check a wide ranger for the devation
1193 | pct_range_med = .1
1194 | price_up = central_price + pct_range_med * central_price 
1195 | price_dn = central_price - pct_range_med * central_price
1196 | price_range = price_up - price_dn
1197 | unused_price, av_dev = find_central_output(output_prices,price_dn,price_up)
1198 | dev_pct = av_dev/price_range
1199 | 
1200 | # use the pct deviation of data to set y axis range
1201 | map_dev_axr = (.15-.05)/(.20-.17)
1202 | ax_range = .05+ (dev_pct-.17)*map_dev_axr
1203 | 
1204 | #set max and min y axis ranges
1205 | if ax_range < .05:
1206 |     ax_range = .05
1207 | if ax_range > .2:
1208 |     ax_range = .2
1209 | price_up = central_price + ax_range * central_price 
1210 | price_dn = central_price - ax_range * central_price
1211 | 
1212 | 
1213 | # print update
1214 | print("100%\t\t\tdone",flush=True)
1215 | if date_mode:
1216 |     print("\n\n\t\t"+price_day_date_utc+" price: $"+f"{int(central_price):,}\n\n",flush=True)
1217 | 
1218 | 
1219 | 
1220 | 
1221 | ##############################################################################
1222 | 
1223 | #  Part 11) Generate the chart in a webpage and serve it to the browser
1224 | 
1225 | ##############################################################################
1226 | 
1227 | # In this final section we use python to create a local webpage and serve that
1228 | # webpage to the local browswer. We write a long string of data whose syntax is that
1229 | # of html/javascript. We then pass python data into that string and save it as
1230 | # an html file in the local directory. We then use python webbroswer to serve the
1231 | # html into the local browser.
1232 | 
1233 | # geometric layout of webpage and chart
1234 | width = 1000   # canvas width (px)
1235 | height = 660  # canvas height (px)
1236 | margin_left = 120
1237 | margin_right = 90
1238 | margin_top = 100
1239 | margin_bottom = 120
1240 | 
1241 | # #get linear block hieghts on x axis
1242 | start = block_nums_needed[0]
1243 | end = block_nums_needed[-1]
1244 | count = len(output_prices)
1245 | step = (end - start) / (count - 1) if count > 1 else 0
1246 | b3 = [start + i * step for i in range(count)]
1247 | 
1248 | #Prepare python prices, heights, and timestamps for sending to html
1249 | heights = []
1250 | heights_smooth = []
1251 | timestamps = []
1252 | prices = []
1253 | for i in range(len(output_prices)):
1254 |     if price_dn < output_prices[i] < price_up:
1255 |         heights.append(output_blocks[i])
1256 |         heights_smooth.append(b3[i])
1257 |         timestamps.append(output_times[i])
1258 |         prices.append(output_prices[i])
1259 | 
1260 | # Sort by timestamps (optional, looks nicer)
1261 | heights_smooth, prices = zip(*sorted(zip(heights_smooth, prices)))
1262 | 
1263 | # set x tick locations
1264 | num_ticks = 5
1265 | n = len(heights_smooth)
1266 | tick_indxs = [round(i * (n - 1) / (num_ticks - 1)) for i in range(num_ticks)]
1267 | 
1268 | # Set the x tick labels 
1269 | xtick_positions = []
1270 | xtick_labels = []
1271 | for tk in tick_indxs:
1272 |     xtick_positions.append(heights_smooth[tk])
1273 |     block_height = heights[tk]
1274 |     timestamp = timestamps[tk]
1275 |     dt = datetime.utcfromtimestamp(timestamp)
1276 |     time_label = f"{dt.hour:02}:{dt.minute:02} UTC"
1277 |     label = f"{block_height}\n{time_label}"  # comma after 3 digits
1278 |     xtick_labels.append(label)
1279 | 
1280 | # Calculate avg price
1281 | avg_price = central_price
1282 | 
1283 | #set the plot annotations
1284 | plot_title_left = ""
1285 | plot_title_right = ""
1286 | bottom_note1 = ""
1287 | bottom_note2 = ""
1288 | if date_mode:
1289 |     plot_title_left = price_day_date_utc+" blocks from local node"
1290 |     plot_title_right ="UTXOracle Consensus Price $"+f"{int(central_price):,} "#+test_price
1291 |     bottom_note1 = "Consensus Data:"
1292 |     bottom_note2 = "this plot is identical and immutable for every bitcoin node"
1293 | if block_mode:
1294 |     plot_title_left = "Local Node Blocks "+str(block_start_num)+"-"+str(block_finish_num)
1295 |     plot_title_right ="UTXOracle Block Window Price $"+f"{int(central_price):,}"
1296 |     bottom_note1 = "* Block Window Price "
1297 |     bottom_note2 = "may have node dependent differences data on the chain tip"
1298 | 
1299 | 
1300 | # Write the HTML code for the chart
1301 | html_content = f'''<!DOCTYPE html>
1302 | 
1303 | <html>
1304 | <head>
1305 | <title>UTXOracle Local</title>
1306 | <style>
1307 |     body {{
1308 |         background-color: black;
1309 |         margin: 0;
1310 |         color: #CCCCCC;
1311 |         font-family: Arial, sans-serif;
1312 |         text-align: center;
1313 |     }}
1314 |     canvas {{
1315 |         background-color: black;
1316 |         display: block;
1317 |         margin: auto;
1318 |     }}
1319 |     
1320 |     
1321 | </style>
1322 | </head>
1323 | <body>
1324 | 
1325 | 
1326 | 
1327 | <div id="tooltip" style="
1328 |     position: absolute;
1329 |     background-color: black;
1330 |     color: cyan;
1331 |     border: 1px solid cyan;
1332 |     padding: 8px;
1333 |     font-size: 14px;
1334 |     border-radius: 5px;
1335 |     pointer-events: none;
1336 |     opacity: 0;
1337 |     transition: opacity 0.2s;
1338 |     text-align: left;
1339 |     z-index: 10;
1340 | "></div>
1341 | 
1342 | <div style="position: relative; width: 95%; max-width: 1000px; margin: auto;">
1343 |     
1344 |     <canvas id="myCanvas" style="width: 100%; height: auto;" width="{width}" height="{height}"></canvas>
1345 |     
1346 |     <button id="downloadBtn" style="
1347 |         position: absolute;
1348 |         bottom: 5%;
1349 |         right: 2%;
1350 |         font-size: 14px;
1351 |         padding: 6px 10px;
1352 |         background-color: black;
1353 |         color: white;
1354 |         border: none;
1355 |         border-radius: 5px;
1356 |         cursor: pointer;
1357 |         opacity: 0.85;
1358 |     ">
1359 |     Save PNG
1360 |     </button>
1361 | 
1362 | </div>
1363 | 
1364 | 
1365 | 
1366 | <script>
1367 | const canvas = document.getElementById('myCanvas');
1368 | const ctx = canvas.getContext('2d');
1369 | 
1370 | const width = {width};
1371 | const height = {height};
1372 | 
1373 | const marginLeft = {margin_left};
1374 | const marginRight = {margin_right};
1375 | const marginTop = {margin_top};
1376 | const marginBottom = {margin_bottom};
1377 | 
1378 | const plotWidth = width - marginLeft - marginRight;
1379 | const plotHeight = height - marginTop - marginBottom;
1380 | 
1381 | // Data
1382 | const heights_smooth = {list(heights_smooth)};
1383 | const prices = {list(prices)};
1384 | const heights = {list(heights)};
1385 | const timestamps = {list(timestamps)};
1386 | 
1387 | 
1388 | const ymin = Math.min(...prices);
1389 | const ymax = Math.max(...prices);
1390 | const xmin = Math.min(...heights_smooth);
1391 | const xmax = Math.max(...heights_smooth);
1392 | 
1393 | // xticks
1394 | const xtick_positions = {xtick_positions};
1395 | const xtick_labels = {xtick_labels};
1396 | 
1397 | 
1398 | // Scaling functions
1399 | function scaleX(t) {{
1400 |     return marginLeft + (t - xmin) / (xmax - xmin) * plotWidth;
1401 | }}
1402 | 
1403 | function scaleY(p) {{
1404 |     return marginTop + (1 - (p - ymin) / (ymax - ymin)) * plotHeight;
1405 | }}
1406 | 
1407 | // Background
1408 | ctx.fillStyle = "black";
1409 | ctx.fillRect(0, 0, width, height);
1410 | 
1411 | 
1412 | // UTXOracle Local Title
1413 | ctx.font = "bold 36px Arial";
1414 | ctx.textAlign = "center";
1415 | ctx.fillStyle = "cyan";
1416 | ctx.fillText("UTXOracle", width / 2 - 60, 40);  // Slight left shift for spacing
1417 | 
1418 | ctx.fillStyle = "lime";
1419 | ctx.fillText("Local", width / 2 + 95, 40);  // Slight right shift
1420 | 
1421 | // Plot Date and Consensus Price
1422 | ctx.font = "24px Arial";
1423 | ctx.textAlign = "right";
1424 | ctx.fillStyle = "white";
1425 | ctx.fillText("{plot_title_left}", width /2, 80);
1426 | ctx.textAlign = "left";
1427 | ctx.fillStyle = "lime";
1428 | ctx.fillText("{plot_title_right}", width/2 +10, 80);
1429 | 
1430 | 
1431 | 
1432 | 
1433 | 
1434 | // Draw axes
1435 | ctx.strokeStyle = "white";
1436 | ctx.lineWidth = 1;
1437 | 
1438 | // Y axis
1439 | ctx.beginPath();
1440 | ctx.moveTo(marginLeft, marginTop);
1441 | ctx.lineTo(marginLeft, marginTop + plotHeight);
1442 | ctx.stroke();
1443 | 
1444 | // X axis
1445 | ctx.beginPath();
1446 | ctx.moveTo(marginLeft, marginTop + plotHeight);
1447 | ctx.lineTo(marginLeft + plotWidth, marginTop + plotHeight);
1448 | ctx.stroke();
1449 | 
1450 | // Right spine
1451 | ctx.beginPath();
1452 | ctx.moveTo(marginLeft + plotWidth, marginTop);
1453 | ctx.lineTo(marginLeft + plotWidth, marginTop + plotHeight);
1454 | ctx.stroke();
1455 | 
1456 | // Top spine
1457 | ctx.beginPath();
1458 | ctx.moveTo(marginLeft, marginTop);
1459 | ctx.lineTo(marginLeft + plotWidth, marginTop);
1460 | ctx.stroke();
1461 | 
1462 | 
1463 | // Draw ticks and labels
1464 | ctx.fillStyle = "white";
1465 | ctx.font = "20px Arial";
1466 | 
1467 | // Y axis ticks
1468 | const yticks = 5;
1469 | for (let i = 0; i <= yticks; i++) {{
1470 |     let p = ymin + (ymax - ymin) * i / yticks;
1471 |     let y = scaleY(p);
1472 |     ctx.beginPath();
1473 |     ctx.moveTo(marginLeft - 5, y);
1474 |     ctx.lineTo(marginLeft, y);
1475 |     ctx.stroke();
1476 |     ctx.textAlign = "right";
1477 |     ctx.fillText(Math.round(p).toLocaleString(), marginLeft - 10, y + 4);
1478 | }}
1479 | 
1480 | // X axis ticks
1481 | ctx.textAlign = "center";
1482 | ctx.font = "16px Arial";
1483 | 
1484 | for (let i = 0; i < xtick_positions.length; i++) {{
1485 |     let x = scaleX(xtick_positions[i]);
1486 |     ctx.beginPath();
1487 |     ctx.moveTo(x, marginTop + plotHeight);
1488 |     ctx.lineTo(x, marginTop + plotHeight + 5);
1489 |     ctx.stroke();
1490 | 
1491 |     // Split label into two lines
1492 |     let parts = xtick_labels[i].split("\\n");
1493 |     ctx.fillText(parts[0], x, marginTop + plotHeight + 20);   // Block height
1494 |     ctx.fillText(parts[1], x, marginTop + plotHeight + 40);   // Time
1495 | }}
1496 | 
1497 | 
1498 | // Axis labels
1499 | ctx.fillStyle = "white";
1500 | ctx.font = "20px Arial";
1501 | ctx.textAlign = "center";
1502 | ctx.fillText("Block Height and UTC Time", marginLeft + plotWidth/2, height - 48);
1503 | ctx.save();
1504 | ctx.translate(20, marginTop + plotHeight/2);
1505 | ctx.rotate(-Math.PI / 2);
1506 | ctx.fillText("BTC Price ($)", 0, 0);
1507 | ctx.restore();
1508 | 
1509 | // Plot points
1510 | ctx.fillStyle = "cyan";
1511 | for (let i = 0; i < heights_smooth.length; i++) {{
1512 |     let x = scaleX(heights_smooth[i]);
1513 |     let y = scaleY(prices[i]);
1514 |     ctx.fillRect(x, y, .75, .75);
1515 | }}
1516 | 
1517 | // Annotation for average price
1518 | ctx.fillStyle = "cyan";
1519 | ctx.font = "20px Arial";
1520 | ctx.textAlign = "left";
1521 | ctx.fillText("- {int(avg_price):,}", marginLeft + plotWidth + 1, scaleY({avg_price}) +0);
1522 | 
1523 | 
1524 | // Annotate bottom chart note
1525 | 
1526 | ctx.font = "24px Arial";
1527 | ctx.fillStyle = "lime";
1528 | ctx.textAlign = "right";
1529 | ctx.fillText("{bottom_note1}", 320, height-10);
1530 | ctx.font = "24px Arial";
1531 | ctx.fillStyle = "white";
1532 | ctx.textAlign = "left";
1533 | ctx.fillText("{bottom_note2}", 325, height-10);
1534 | 
1535 | 
1536 | // === MOUSEOVER INFO ===
1537 | 
1538 | const tooltip = document.getElementById('tooltip');
1539 | 
1540 | canvas.addEventListener('mousemove', function(event) {{
1541 | const rect = canvas.getBoundingClientRect();
1542 | const scaleX = canvas.width / rect.width;
1543 | const scaleY = canvas.height / rect.height;
1544 | const mouseX = (event.clientX - rect.left) * scaleX;
1545 | const mouseY = (event.clientY - rect.top) * scaleY;
1546 | 
1547 | if (mouseX >= marginLeft && mouseX <= width - marginRight &&
1548 |     mouseY >= marginTop && mouseY <= marginTop + plotHeight) {{
1549 | 
1550 |     const fractionAcross = (mouseX - marginLeft) / plotWidth;
1551 |     let index = Math.round(fractionAcross * (heights.length - 1));
1552 |     index = Math.max(0, Math.min(index, heights.length - 1));
1553 | 
1554 |     const price = ymax - (mouseY - marginTop) / plotHeight * (ymax - ymin);
1555 |     const blockHeight = heights[index];
1556 |     const timestamp = timestamps[index];
1557 | 
1558 |     const date = new Date(timestamp * 1000);
1559 |     const hours = date.getUTCHours().toString().padStart(2, '0');
1560 |     const minutes = date.getUTCMinutes().toString().padStart(2, '0');
1561 |     const utcTime = `${{hours}}:${{minutes}} UTC`;
1562 | 
1563 |     tooltip.innerHTML = `
1564 |         Price: $${{Math.round(price).toLocaleString()}}<br>
1565 |         Block: ${{blockHeight.toLocaleString()}}<br>
1566 |         Time: ${{utcTime}}
1567 |     `;
1568 | 
1569 |     tooltip.style.left = (event.clientX + 5) + 'px';
1570 |     tooltip.style.top = (event.clientY + window.scrollY - 75) + 'px';
1571 |     tooltip.style.opacity = 1;
1572 | }} else {{
1573 |     tooltip.style.opacity = 0;
1574 | }}
1575 | }});
1576 | 
1577 | 
1578 | 
1579 | 
1580 | </script>
1581 | 
1582 | 
1583 | <script>
1584 | // Download canvas as PNG
1585 | const downloadBtn = document.getElementById('downloadBtn');
1586 | downloadBtn.addEventListener('click', function() {{
1587 |     const link = document.createElement('a');
1588 |     link.download = 'UTXOracle_Local_Node_Price.png';
1589 |     link.href = canvas.toDataURL('image/png');
1590 |     link.click();
1591 | }});
1592 | </script>
1593 | 
1594 | <br>
1595 | <br>
1596 | 
1597 | 
1598 | <h2 style="margin-top:10px; font-size:24px;">
1599 | Want a 
1600 | <span style="color:orange; font-size:24px;">Live Updating Oracle</span>
1601 |   with New Mempool Transactions? 
1602 | </h2>
1603 | 
1604 | 
1605 | <iframe
1606 |     width="{width}"
1607 |     height="{height}"
1608 |     src="https://www.youtube.com/embed/live_stream?channel=UCXN7Xa_BF7dqLErzOmS-B7Q&autoplay=1&mute=1"
1609 |     frameborder="0"
1610 |     allow="autoplay; encrypted-media"
1611 |     allowfullscreen
1612 |     
1613 | </iframe>
1614 | 
1615 | 
1616 | 
1617 | <br>
1618 | 
1619 | </body>
1620 | </html>
1621 | '''
1622 | 
1623 | 
1624 | 
1625 | 
1626 | # name the file with dates or blocks
1627 | filename = ".html"
1628 | if date_mode:
1629 |     filename = "UTXOracle_"+price_date_dash+filename
1630 | if block_mode:
1631 |     filename = "UTXOracle_"+str(block_start_num)+"-"+str(block_finish_num)+filename
1632 | 
1633 | 
1634 | # Write file locally and serve to browser
1635 | import webbrowser
1636 | with open(filename, "w") as f:
1637 |     f.write(html_content)
1638 | webbrowser.open('file://' + os.path.realpath(filename))
1639 | 
1640 | 
1641 | 
1642 | 
1643 | 
1644 | 


--------------------------------------------------------------------------------
/UTXOracle.py:
--------------------------------------------------------------------------------
   1 | 
   2 | #########################################################################################  
   3 | #                                                                                       #
   4 | #   /$$   /$$ /$$$$$$$$ /$$   /$$  /$$$$$$                               /$$            #
   5 | #  | $$  | $$|__  $$__/| $$  / $$ /$$__  $$                             | $$            #
   6 | #  | $$  | $$   | $$   |  $$/ $$/| $$  \ $$  /$$$$$$  /$$$$$$   /$$$$$$$| $$  /$$$$$$   #
   7 | #  | $$  | $$   | $$    \  $$$$/ | $$  | $$ /$$__  $$|____  $$ /$$_____/| $$ /$$__  $$  #
   8 | #  | $$  | $$   | $$     >$$  $$ | $$  | $$| $$  \__/ /$$$$$$$| $$      | $$| $$$$$$$$  #
   9 | #  | $$  | $$   | $$    /$$/\  $$| $$  | $$| $$      /$$__  $$| $$      | $$| $$_____/  #
  10 | #  |  $$$$$$/   | $$   | $$  \ $$|  $$$$$$/| $$     |  $$$$$$$|  $$$$$$$| $$|  $$$$$$$  #
  11 | #   \______/    |__/   |__/  |__/ \______/ |__/      \_______/ \_______/|__/ \_______/  #
  12 | #                                                                                       #
  13 | #########################################################################################  
  14 | #                     Version 9.1 RPC Only
  15 | 
  16 | 
  17 | 
  18 | 
  19 | 
  20 | 
  21 | ###############################################################################  
  22 | 
  23 | #  Introduction
  24 | 
  25 | ###############################################################################  
  26 | 
  27 |  # Thank you for taking the time to open this. Computer programs have two functions:
  28 |  # one for the computer and one for the human. This might be the first time you've
  29 |  # attempted to read a computer program. If so, these lines starting with the hash tag
  30 |  # are for you. They’ll tell you what the code is doing at all times. The non–hash-tag
  31 |  # lines are used by the computer, though if you spend a few seconds with them, 
  32 |  # you’ll be able to understand exactly what they’re doing.
  33 | 
  34 |  # Since you opened this, you probably already have a good idea of what UTXOracle
  35 |  # does. It finds the price of Bitcoin using data only from your own node. It doesn’t
  36 |  # contact any third parties. The code also has no third-party dependencies. It runs
  37 |  # on the most basic version of Python 3 that you can find. Many operating systems
  38 |  # have Python 3 built in, but many don’t. You’ll also need to know how to open a
  39 |  # terminal window on your computer. To see if you already have Python 3 installed,
  40 |  # open a terminal window and type python3 --version, then hit Enter. If it doesn’t
  41 |  # show you a version number, you’ll need to download and install it from python.org.
  42 | 
  43 |  # If you’re reading this from a file on your computer, then you already have the
  44 |  # program. If you’re reading this off a website, you need to download it to your
  45 |  # computer. You can do this with the simple command "curl -O https://utxo.live/oracle/UTXOracle.py".
  46 |  # It will download this single file as UTXOracle.py. Then run the program with
  47 |  # "python3 UTXOracle.py" which will find yesterday's price by default. To run the program
  48 |  # on different days, and up to the latest block, type "python3 UTXOracle.py -h"
  49 |  # to see the command options.
  50 | 
  51 |  # If you’re an experienced coder, you’ll notice many inefficiencies in this code.
  52 |  # It is a single file that runs top to bottom. 
  53 |  # It doesn’t not make use of advanced libraries that would make the code more
  54 |  # efficient at the cost of third party dependence. It repeats code for clarity instead of
  55 |  # defining functions where the user has to constantly scroll up and down and to
  56 |  # other files to see how the function works. The purpose of this code is not for
  57 |  # efficiency, or for corporate team implementations. The purpose is to explain how
  58 |  # the UTXOracle algorithm works side by side with the code.
  59 | 
  60 |  # The code proceeds by completing the following 12 steps in order. Approximate line
  61 |  # numbers of the steps are listed for the user to jump directly there if desired.
  62 |  
  63 | # Step 1 - Configuration Options...................Line 78
  64 | # Step 2 - Establish RPC Connection................Line 206
  65 | # Step 3 - Check Dates.............................Line 315
  66 | # Step 4 - Find Block Hashes.......................Line 409
  67 | # Step 5 - Initialize Histogram....................Line 589
  68 | # Step 6 - Load Histogram from Transaction Data....Line 646
  69 | # Step 7 - Remove Round Bitcoin Amounts............Line 889
  70 | # Step 8 - Construct the Price Finding Stencil.....Line 971
  71 | # Step 9 - Estimate a Rough Price..................Line 1049
  72 | # Step 10 - Create Intraday Price Points...........Line 1160
  73 | # Step 11 - Find the Exact Average Price...........Line 1260
  74 | # Step 12 - Generate a Price Plot HTML Page........Line 1377
  75 | 
  76 | 
  77 | 
  78 | 
  79 | ###############################################################################  
  80 | 
  81 | # Step 1 - Configuation Options
  82 | 
  83 | ############################################################################### 
  84 | 
  85 | # The way UTXOracle connects to your Bitcoin node depends on your operating
  86 | # system and the settings in your bitcoin.conf file. First, the program determines
  87 | # your operating system to make an initial guess at the default location of your
  88 | # primary Bitcoin directory. If you're not using the default directory, use the -p option
  89 | # when running the program to specify the correct block directory.
  90 | 
  91 | # Other options let you specify the historical date you want to run (-h) or request the
  92 | # price from the most recent 144 blocks (-rb). If you're an advanced user with
  93 | # custom settings overriding the default RPC connection, the program will read your
  94 | # bitcoin.conf file to use them. Otherwise, it will use the autogenerated cookie file for
  95 | # authentication.
  96 | 
  97 | 
  98 | # set platform dependent data paths
  99 | import platform
 100 | import os
 101 | data_dir = []
 102 | system = platform.system()
 103 | if system == "Darwin":  # macOS
 104 |     data_dir = os.path.expanduser("~/Library/Application Support/Bitcoin")
 105 | elif system == "Windows":
 106 |     data_dir = os.path.join(os.environ.get("APPDATA", ""), "Bitcoin")
 107 | else:  # Linux or others
 108 |     data_dir = os.path.expanduser("~/.bitcoin")
 109 | 
 110 | # initialize variables for blocks and dates
 111 | date_entered = ""
 112 | date_mode = True
 113 | block_mode = False
 114 | block_start_num = 0
 115 | block_finish_num = 0
 116 | block_nums_needed = []
 117 | block_hashes_needed = []
 118 | block_times_needed = []
 119 | 
 120 | #print help text for the user if needed
 121 | import sys
 122 | def print_help():
 123 |     help_text = """
 124 | Usage: python3 UTXOracle.py [options]
 125 | 
 126 | Options:
 127 |   -h               Show this help message
 128 |   -d YYYY/MM/DD    Specify a UTC date to evaluate
 129 |   -p /path/to/dir  Specify the data directory for blk files
 130 |   -rb              Use last 144 recent blocks instead of date mode
 131 | """
 132 |     print(help_text)
 133 |     sys.exit(0)
 134 | 
 135 | #did use ask for help
 136 | if "-h" in sys.argv:
 137 |     print_help()
 138 |     
 139 | #did user specify a date?
 140 | if "-d" in sys.argv:
 141 |     h_index = sys.argv.index("-d")
 142 |     if h_index + 1 < len(sys.argv):
 143 |         date_entered = sys.argv[h_index + 1]
 144 | 
 145 | #did user specify a data path?
 146 | if "-p" in sys.argv:
 147 |     d_index = sys.argv.index("-p")
 148 |     if d_index + 1 < len(sys.argv):
 149 |         data_dir = sys.argv[d_index + 1]
 150 | 
 151 | #did user specify blocks instead of a date?
 152 | if "-rb" in sys.argv:
 153 |     date_mode = False
 154 |     block_mode = True
 155 | 
 156 | # Look for bitcoin.conf or bitcoin_rw.conf
 157 | conf_path = None
 158 | conf_candidates = ["bitcoin.conf", "bitcoin_rw.conf"]
 159 | for fname in conf_candidates:
 160 |     path = os.path.join(data_dir, fname)
 161 |     if os.path.exists(path):
 162 |         conf_path = path
 163 |         break
 164 | if not conf_path:
 165 |     print(f"Invalid Bitcoin data directory: {data_dir}")
 166 |     print("Expected to find 'bitcoin.conf' or 'bitcoin_rw.conf' in this directory.")
 167 |     sys.exit(1)
 168 | 
 169 | # Parse the conf file
 170 | conf_settings = {}
 171 | with open(conf_path, 'r') as f:
 172 |     for line in f:
 173 |         line = line.strip()
 174 |         if not line or line.startswith("#"):
 175 |             continue
 176 |         if "=" in line:
 177 |             key, value = line.split("=", 1)
 178 |             conf_settings[key.strip()] = value.strip().strip('"')
 179 | 
 180 | # Set blocks directory
 181 | blocks_dir = os.path.expanduser(conf_settings.get("blocksdir", os.path.join(data_dir, "blocks")))
 182 | 
 183 | # Prepare RPC parameters
 184 | rpc_user = conf_settings.get("rpcuser")
 185 | rpc_password = conf_settings.get("rpcpassword")
 186 | cookie_path = conf_settings.get("rpccookiefile", os.path.join(data_dir, ".cookie"))
 187 | rpc_host = conf_settings.get("rpcconnect", "127.0.0.1")
 188 | rpc_port = int(conf_settings.get("rpcport", "8332"))
 189 | 
 190 | # Prepare bitcoin-cli fallback options (if needed elsewhere)
 191 | bitcoin_cli_options = []
 192 | if rpc_user and rpc_password:
 193 |     bitcoin_cli_options.append(f"-rpcuser={rpc_user}")
 194 |     bitcoin_cli_options.append(f"-rpcpassword={rpc_password}")
 195 | else:
 196 |     if os.path.exists(cookie_path):
 197 |         bitcoin_cli_options.append(f"-rpccookiefile={cookie_path}")
 198 | 
 199 | if rpc_host:
 200 |     bitcoin_cli_options.append(f"-rpcconnect={rpc_host}")
 201 | if "rpcport" in conf_settings:
 202 |     bitcoin_cli_options.append(f"-rpcport={conf_settings['rpcport']}")
 203 | 
 204 | 
 205 | 
 206 | 
 207 | ###############################################################################  
 208 | 
 209 | #  Step 2 - Establish RPC Connection
 210 | 
 211 | ###############################################################################  
 212 | 
 213 | # Community consensus has established that RPC should be the standard front door
 214 | # through which other programs communicate with your Bitcoin node. Since we'll be
 215 | # making repeated requests to the node, we define a general function that attaches
 216 | # the necessary RPC credentials to any command we send and returns the
 217 | # response to where it's needed in the program. If this is the first time you've used
 218 | # RPC, the function will create the RPC cookie file using the node's default method.
 219 | # Otherwise, it simply returns the result of the command and prints any errors
 220 | # encountered. After defining the Ask_Node function, we test it by requesting the
 221 | # latest block the node has received.
 222 | 
 223 | 
 224 | print("\nCurrent operation  \t\t\t\tTotal Completion",flush=True)
 225 | print("\nConnecting to node...",flush=True)
 226 | print("0%..", end="",flush=True)
 227 | 
 228 | # define the node communication function
 229 | import http.client
 230 | import json
 231 | import base64
 232 | 
 233 | def Ask_Node(command, cred_creation):
 234 |     method = command[0]
 235 |     params = command[1:]
 236 | 
 237 |     # Handle authentication
 238 |     rpc_u = rpc_user
 239 |     rpc_p = rpc_password
 240 | 
 241 |     if not rpc_u or not rpc_p:
 242 |         try:
 243 |             with open(cookie_path, "r") as f:
 244 |                 cookie = f.read().strip()
 245 |                 rpc_u, rpc_p = cookie.split(":", 1)
 246 |         except Exception as e:
 247 |             print("Error reading .cookie file for RPC authentication.")
 248 |             print("Details:", e)
 249 |             sys.exit(1)
 250 | 
 251 |     # Prepare JSON-RPC payload
 252 |     payload = json.dumps({
 253 |         "jsonrpc": "1.0",
 254 |         "id": "utxoracle",
 255 |         "method": method,
 256 |         "params": params
 257 |     })
 258 | 
 259 |     # Basic auth header
 260 |     auth_header = base64.b64encode(f"{rpc_u}:{rpc_p}".encode()).decode()
 261 |     headers = {
 262 |         "Content-Type": "application/json",
 263 |         "Authorization": f"Basic {auth_header}"
 264 |     }
 265 | 
 266 |     try:
 267 |         conn = http.client.HTTPConnection(rpc_host, rpc_port)
 268 |         conn.request("POST", "/", payload, headers)
 269 |         response = conn.getresponse()
 270 |         if response.status != 200:
 271 |             raise Exception(f"HTTP error {response.status} {response.reason}")
 272 |         raw_data = response.read()
 273 |         conn.close()
 274 | 
 275 |         # Extract result and mimic `subprocess.check_output` return (as bytes)
 276 |         parsed = json.loads(raw_data)
 277 |         if parsed.get("error"):
 278 |             raise Exception(parsed["error"])
 279 |         result = parsed["result"]
 280 |         if isinstance(result, (dict, list)):
 281 |             return json.dumps(result, indent=2).encode()  # pretty-print like CLI
 282 |         else:
 283 |             return str(result).encode()
 284 | 
 285 |     except Exception as e:
 286 |         if not cred_creation:
 287 |             print("Error connecting to your node via RPC. Troubleshooting steps:\n")
 288 |             print("\t1) Ensure bitcoind or bitcoin-qt is running with server=1.")
 289 |             print("\t2) Check rpcuser/rpcpassword or .cookie.")
 290 |             print("\t3) Verify RPC port/host settings.")
 291 |             print("\nThe attempted RPC method was:", method)
 292 |             print("Parameters:", params)
 293 |             print("\nThe error was:\n", e)
 294 |             sys.exit(1)
 295 | 
 296 | 
 297 | #get current block height from local node and exit if connection not made
 298 | print("20%..", end="",flush=True)
 299 | Ask_Node(['getblockcount'], True) #create RPC creds if necessary
 300 | block_count_b = Ask_Node(['getblockcount'], False)
 301 | print("40%..", end="",flush=True)
 302 | block_count = int(block_count_b)             #convert text to integer
 303 | block_count_consensus = block_count-6
 304 | 
 305 | #get block header from current block height
 306 | #block_hash = Ask_Node(['getblockhash', str(block_count_consensus)]).decode().strip()
 307 | block_hash = Ask_Node(['getblockhash',block_count_consensus], False).decode().strip()
 308 | print("60%..", end="",flush=True)
 309 | block_header_b = Ask_Node(['getblockheader', block_hash, True], False)
 310 | block_header = json.loads(block_header_b)
 311 | print("80%..", end="",flush=True)
 312 | 
 313 | 
 314 | 
 315 | 
 316 | ###############################################################################  
 317 | 
 318 | # Step 3 - Check Dates  
 319 | 
 320 | ############################################################################### 
 321 | 
 322 | # The Bitcoin price does not exist instantaneously on-chain. A single block often
 323 | # does not contain enough transaction data to determine the price—and in some
 324 | # cases, may contain no transactions at all. For this reason, a time-averaging
 325 | # window is required to accumulate sufficient data to estimate the price accurately.
 326 | 
 327 | # While many averaging windows could work, a single day is both a natural human
 328 | # time scale and typically includes enough transaction activity. For accounting and
 329 | # historical purposes, daily resolution is also a commonly used standard. Therefore, 
 330 | # UTXOracle uses the daily average as its default time window for determining price.
 331 | 
 332 | # This means UTXOracle must know two things: (1) the most recent date covered by 
 333 | # the node’s block data, and (2) the date the user is requesting the price for. If the 
 334 | # requested date is too far in the past or future relative to the available data, an error 
 335 | # message will be displayed. The earliest supported date depends on how far back 
 336 | # the current version of UTXOracle has been tested.
 337 | 
 338 | 
 339 | 
 340 | #import built in tools for dates/times and json style lists
 341 | from datetime import datetime, timezone, timedelta
 342 | 
 343 | 
 344 | 
 345 | #get the date and time of the current block height
 346 | latest_time_in_seconds = block_header['time']
 347 | time_datetime = datetime.fromtimestamp(latest_time_in_seconds,tz=timezone.utc)
 348 | 
 349 | #get the date/time of utc midnight on the latest day
 350 | latest_year  = int(time_datetime.strftime("%Y"))
 351 | latest_month = int(time_datetime.strftime("%m"))
 352 | latest_day   = int(time_datetime.strftime("%d"))
 353 | latest_utc_midnight = datetime(latest_year,latest_month,latest_day,0,0,0,tzinfo=timezone.utc)
 354 | 
 355 | #assign the day before as the latest possible price date
 356 | seconds_in_a_day = 60*60*24
 357 | yesterday_seconds = latest_time_in_seconds - seconds_in_a_day
 358 | latest_price_day = datetime.fromtimestamp(yesterday_seconds,tz=timezone.utc)
 359 | latest_price_date = latest_price_day.strftime("%Y-%m-%d")
 360 | print("100%", end="",flush=True)
 361 | 
 362 | #print completion update
 363 | print("\t\t\t5% done",flush=True)
 364 | 
 365 | 
 366 | # If running in date mode, make sure that the date requested is in the
 367 | if date_mode:
 368 | 
 369 |     # run latest day if hit enter
 370 |     if (date_entered == ""):
 371 |         datetime_entered = latest_utc_midnight + timedelta(days=-1)
 372 |         
 373 |     #user entered a specific date
 374 |     else:
 375 |         
 376 |         #check to see if this is a good date
 377 |         try:
 378 |             year  = int(date_entered.split('/')[0])
 379 |             month = int(date_entered.split('/')[1])
 380 |             day = int(date_entered.split('/')[2])
 381 |             
 382 |             #make sure this date is less than the max date
 383 |             datetime_entered = datetime(year,month,day,0,0,0,tzinfo=timezone.utc)
 384 |             if datetime_entered.timestamp() >= latest_utc_midnight.timestamp():
 385 |                 print("\nDate is after the latest avaiable. We need 6 blocks after UTC midnight.")
 386 |                 print("Run UTXOracle.py -rb for the most recent blocks")
 387 |                 
 388 |                 sys.exit()
 389 |             
 390 |             #make sure this date is after the min date
 391 |             dec_15_2023 = datetime(2023,12,15,0,0,0,tzinfo=timezone.utc)
 392 |             if datetime_entered.timestamp() < dec_15_2023.timestamp():
 393 |                 print("\nThe date entered is before 2023-12-15, please try again")
 394 |                 sys.exit()
 395 |         
 396 |         except:
 397 |             print("\nError interpreting date. Please try again. Make sure format is YYYY/MM/DD")
 398 |             sys.exit()
 399 |     
 400 |     
 401 |     #get the seconds and printable date string of date entered
 402 |     price_day_seconds = int(datetime_entered.timestamp())
 403 |     price_day_date_utc = datetime_entered.strftime("%b %d, %Y")
 404 |     price_date_dash = datetime_entered.strftime("%Y-%m-%d")
 405 |     
 406 | 
 407 | 
 408 | 
 409 | 
 410 | ##############################################################################  
 411 | 
 412 | # Step 4 - Find Block Hashes
 413 | 
 414 | ##############################################################################  
 415 | 
 416 | # Bitcoin nodes don’t store blocks by date—or even by block height. Instead, they 
 417 | # store blocks by their hash. This has two implications: first, we need to know which 
 418 | # block heights we're looking for, and second, we must retrieve the corresponding 
 419 | # block hashes for those heights.
 420 | 
 421 | # The task is simpler when running in block mode (-rb), where we determine the 
 422 | # needed blocks by subtracting 144 from the most recent block height. In date mode, 
 423 | # however, the process is more involved, as we must enter a guess-and-check loop 
 424 | # to determine which blocks correspond to the desired calendar day.
 425 | 
 426 | # Since blocks are mined roughly every ten minutes, we can use that as a guideline. 
 427 | # When a user enters a date, we first estimate how many blocks ago that date 
 428 | # occurred. We then check block timestamps starting from our estimate, moving 
 429 | # forward to find the last block of that day, and backward to find the first block.
 430 | 
 431 | # Once we’ve identified all the relevant block heights, we store the hashes of each 
 432 | # block in a list so that we can retrieve them one by one in the next step.
 433 | 
 434 | 
 435 | 
 436 | #define a shortcut for getting the block time from the block number
 437 | def get_block_time(height):
 438 |     block_hash_b = Ask_Node(['getblockhash',height], False)
 439 |     #block_header_b = Ask_Node(['getblockheader',block_hash_b[:64],True])
 440 |     block_header_b = Ask_Node(['getblockheader', block_hash_b.decode().strip(), True], False)
 441 | 
 442 |     block_header = json.loads(block_header_b)
 443 |     return(block_header['time'], block_hash_b[:64].decode())
 444 | 
 445 | #define a shortcut for getting the day of money from a time in seconds
 446 | def get_day_of_month(time_in_seconds):
 447 |     time_datetime = datetime.fromtimestamp(time_in_seconds,tz=timezone.utc)
 448 |     return(int(time_datetime.strftime("%d")))
 449 | 
 450 | #if block mode add the blocks and hashes to a list
 451 | if block_mode:
 452 |     
 453 |     print("\nFinding the last 144 blocks",flush=True)
 454 |     
 455 |     #get the last block number of the day
 456 |     block_finish_num = block_count
 457 |     block_start_num = block_finish_num - 144
 458 |     
 459 |     #append needed block nums and hashes needed
 460 |     block_num = block_start_num
 461 |     time_in_seconds, hash_end = get_block_time(block_start_num)
 462 |     print_every = 0
 463 |     while block_num < block_finish_num:
 464 |         
 465 |         #print update
 466 |         if (block_num-block_start_num)/144*100 > print_every and print_every < 100:
 467 |             print(str(print_every)+"%..",end="",flush=True)
 468 |             print_every += 20
 469 |         block_nums_needed.append(block_num)
 470 |         block_hashes_needed.append(hash_end)
 471 |         block_times_needed.append(time_in_seconds)
 472 |         block_num += 1
 473 |         time_in_seconds, hash_end = get_block_time(block_num)
 474 |         
 475 |     print("100%\t\t\t25% done",flush=True)
 476 | 
 477 | #if date mode search for all the blocks on this day
 478 | elif date_mode:
 479 |     
 480 |     print("\nFinding first blocks on "+datetime_entered.strftime("%b %d, %Y"),flush=True)
 481 |     print("0%..",end="", flush=True)
 482 |     #first estimate of the block height of the price day
 483 |     seconds_since_price_day = latest_time_in_seconds - price_day_seconds
 484 |     blocks_ago_estimate = round(144*float(seconds_since_price_day)/float(seconds_in_a_day))
 485 |     price_day_block_estimate = block_count_consensus - blocks_ago_estimate
 486 |     
 487 |     #check the time of the price day block estimate
 488 |     time_in_seconds, hash_end = get_block_time(price_day_block_estimate) 
 489 |     
 490 |     #get new block estimate from the seconds difference using 144 blocks per day
 491 |     print("20%..",end="",flush=True)
 492 |     seconds_difference = time_in_seconds - price_day_seconds
 493 |     block_jump_estimate = round(144*float(seconds_difference)/float(seconds_in_a_day))
 494 |     
 495 |     #iterate above process until it oscillates around the correct block
 496 |     last_estimate = 0
 497 |     last_last_estimate = 0
 498 |     
 499 |     print("40%..",end="",flush=True)
 500 |     while block_jump_estimate >6 and block_jump_estimate != last_last_estimate:
 501 |         
 502 |         #when we oscillate around the correct block, last_last_estimate = block_jump_estimate
 503 |         last_last_estimate = last_estimate
 504 |         last_estimate = block_jump_estimate
 505 |         
 506 |         #get block header or new estimate
 507 |         price_day_block_estimate = price_day_block_estimate-block_jump_estimate
 508 |         
 509 |         #check time of new block and get new block jump estimate
 510 |         time_in_seconds, hash_end = get_block_time(price_day_block_estimate) 
 511 |         seconds_difference = time_in_seconds - price_day_seconds
 512 |         block_jump_estimate = round(144*float(seconds_difference)/float(seconds_in_a_day))
 513 |     
 514 |     print("60%..",end="",flush=True)
 515 |     #the oscillation may be over multiple blocks so we add/subtract single blocks 
 516 |     #to ensure we have exactly the first block of the target day
 517 |     if time_in_seconds > price_day_seconds:
 518 |         
 519 |         # if the estimate was after price day look at earlier blocks
 520 |         while time_in_seconds > price_day_seconds:
 521 |             
 522 |             #decrement the block by one, read new block header, check time
 523 |             price_day_block_estimate = price_day_block_estimate-1
 524 |             time_in_seconds, hash_end = get_block_time(price_day_block_estimate) 
 525 |             
 526 |         #the guess is now perfectly the first block before midnight
 527 |         price_day_block_estimate = price_day_block_estimate + 1
 528 |         
 529 |     # if the estimate was before price day look for later blocks
 530 |     elif time_in_seconds < price_day_seconds:
 531 |         
 532 |         while time_in_seconds < price_day_seconds:
 533 |             
 534 |             #increment the block by one, read new block header, check time
 535 |             price_day_block_estimate = price_day_block_estimate+1
 536 |             time_in_seconds, hash_end = get_block_time(price_day_block_estimate) 
 537 |     
 538 |     print("80%..",end="",flush=True)
 539 |     #assign the estimate as the price day block since it is correct now    
 540 |     price_day_block = price_day_block_estimate
 541 |     
 542 |     #get the day of the month 
 543 |     time_in_seconds, hash_start = get_block_time(price_day_block)
 544 |     day1 = get_day_of_month(time_in_seconds)
 545 |     
 546 |     #get the last block number of the day
 547 |     price_day_block_end = price_day_block 
 548 |     time_in_seconds, hash_end = get_block_time(price_day_block_end)
 549 |     day2 = get_day_of_month(time_in_seconds)
 550 |     
 551 |     print("100%\t\t\t25% done",flush=True)
 552 |     print("\nFinding last blocks on "+datetime_entered.strftime("%b %d, %Y"),flush=True)
 553 |     
 554 |     #load block nums and hashes needed
 555 |     block_num = 0
 556 |     print_next = 0
 557 |     while day1 == day2:
 558 |         
 559 |         #print progress update
 560 |         block_num+=1
 561 |         if block_num/144 * 100 > print_next:
 562 |             if print_next < 100:
 563 |                 print(str(print_next)+"%..",end="",flush=True)
 564 |                 print_next +=20
 565 |         
 566 |         #append needed block
 567 |         block_nums_needed.append(price_day_block_end)
 568 |         block_hashes_needed.append(hash_end)
 569 |         block_times_needed.append(time_in_seconds)
 570 |         price_day_block_end += 1 #assume 30+ blocks this day
 571 |         time_in_seconds, hash_end = get_block_time(price_day_block_end)
 572 |         day2 = get_day_of_month(time_in_seconds)
 573 |     
 574 |     #complete print update status
 575 |     while print_next<100:
 576 |         print(str(print_next)+"%..",end="",flush=True)
 577 |         print_next +=20
 578 |     
 579 |     #set start and end block numbers
 580 |     block_start_num = price_day_block
 581 |     block_finish_num = price_day_block_end
 582 | 
 583 |     print("100%\t\t\t50% done",flush=True)
 584 | 
 585 | 
 586 | 
 587 | 
 588 | 
 589 | 
 590 | ##############################################################################
 591 | 
 592 | # Step 5 - Initial histogram
 593 | 
 594 | ##############################################################################
 595 | 
 596 | # Just as the price of Bitcoin does not exist at a single instant in time, it also does not 
 597 | # exist at a single satoshi amount. The on-chain price emerges because users tend 
 598 | # to spend Bitcoin in round fiat amounts. To detect this, we create a histogram that 
 599 | # reveals clusters of satoshi values where round fiat amounts are most likely to 
 600 | # occur.
 601 | 
 602 | # We divide the BTC value range into intervals and count how many transaction 
 603 | # outputs fall into each interval. This allows us to visualize where spending patterns 
 604 | # cluster. If the intervals are too small, the data becomes noisy; if they’re too large, 
 605 | # important detail is lost. A rough estimate of the ideal interval width is the average 
 606 | # daily fiat volatility, which we’ve found to be around 0.5%—corresponding to roughly 
 607 | # 200 intervals between each power of ten in BTC.
 608 | 
 609 | # We must also define the upper and lower bounds of the histogram: the smallest 
 610 | # and largest BTC amounts likely to capture typical round fiat values. This range will 
 611 | # evolve as Bitcoin’s purchasing power changes and may need to be updated in 
 612 | # future versions of UTXOracle. From 2020 to 2025, we’ve found that most round fiat 
 613 | # amounts fall within the range of 10^-6 to 10^6 BTC.
 614 | 
 615 | # Once the range and interval size are established, we generate arrays representing 
 616 | # the edges of each interval and initialize a corresponding array of zeros to count 
 617 | # how many transaction outputs fall into each bucket.
 618 | 
 619 | 
 620 | # Define the maximum and minimum values (in log10) of btc amounts to use
 621 | first_bin_value = -6
 622 | last_bin_value = 6  #python -1 means last in list
 623 | range_bin_values = last_bin_value - first_bin_value 
 624 | 
 625 | # create a list of output_histogram_bins and add zero sats as the first bin
 626 | output_histogram_bins = [0.0] #a decimal tells python the list will contain decimals
 627 | 
 628 | # calculate btc amounts of 200 samples in every 10x from 100 sats (1e-6 btc) to 100k (1e5) btc
 629 | for exponent in range(-6,6): #python range uses 'less than' for the big number 
 630 |     
 631 |     #add 200 bin_width increments in this 10x to the list
 632 |     for b in range(0,200):
 633 |         
 634 |         bin_value = 10 ** (exponent + b/200)
 635 |         output_histogram_bins.append(bin_value)
 636 | 
 637 | # Create a list the same size as the bell curve to keep the count of the bins
 638 | number_of_bins = len(output_histogram_bins)
 639 | output_histogram_bin_counts = []
 640 | for n in range(0,number_of_bins):
 641 |     output_histogram_bin_counts.append(float(0.0))
 642 | 
 643 | 
 644 | 
 645 | 
 646 | 
 647 | ##############################################################################
 648 | 
 649 | #  Step 6 -  Load Transaction Data
 650 | 
 651 | ##############################################################################
 652 | 
 653 | # This section of the algorithm is the most time-consuming because it reads every 
 654 | # transaction from the range of blocks requested by the user. We typically need 
 655 | # around 144 blocks, and since each block is roughly 1MB, this means processing 
 656 | # about 144 MB of data.
 657 | 
 658 | # To improve efficiency, we request the raw binary block data and manually convert it 
 659 | # into integers and strings. This requires defining functions that translate binary data 
 660 | # into integers, such as read_varint and encode_varint. We also compute the txid 
 661 | # manually, since binary Bitcoin blocks do not store it directly.
 662 | 
 663 | # After defining these functions, we loop through the list of required block hashes 
 664 | # and extract transaction output amounts to place into histogram bins. We apply 
 665 | # several filters to exclude transactions that are unlikely to reflect meaningful price 
 666 | # information.
 667 | 
 668 | # Through years of testing we decided to filter out transactions containing: more than 
 669 | # 5 inputs, more than 2 outputs, coinbase outputs, op_return outputs, large witness 
 670 | # scripts, and same day inputs. If the output passes the filters, it is inserted into the 
 671 | # histogram bin according to the bitcoin amount of the output.
 672 | 
 673 | 
 674 | print("\nLoading every transaction from every block",flush=True)
 675 | 
 676 | 
 677 | 
 678 | # #initialize output lists and variables
 679 | from struct import unpack
 680 | import binascii
 681 | todays_txids = set()
 682 | raw_outputs = []
 683 | block_heights_dec = []
 684 | block_times_dec = []
 685 | print_next = 0
 686 | block_num = 0
 687 | 
 688 | 
 689 | #shortcut for reading bytes of data from the block file
 690 | import struct
 691 | from math import log10 
 692 | import hashlib
 693 | def read_varint(f):
 694 |     i = f.read(1)
 695 |     if not i:
 696 |         return 0
 697 |     i = i[0]
 698 |     if i < 0xfd:
 699 |         return i
 700 |     elif i == 0xfd:
 701 |         val = struct.unpack("<H", f.read(2))[0]
 702 |     elif i == 0xfe:
 703 |         val = struct.unpack("<I", f.read(4))[0]
 704 |     else:
 705 |         val = struct.unpack("<Q", f.read(8))[0]
 706 |     return val
 707 | 
 708 | #shortcut for encoding variable size integers to bytes
 709 | from io import BytesIO
 710 | def encode_varint(i: int) -> bytes:
 711 |     assert i >= 0
 712 |     if i < 0xfd:
 713 |         return i.to_bytes(1, 'little')
 714 |     elif i <= 0xffff:
 715 |         return b'\xfd' + i.to_bytes(2, 'little')
 716 |     elif i <= 0xffffffff:
 717 |         return b'\xfe' + i.to_bytes(4, 'little')
 718 |     else:
 719 |         return b'\xff' + i.to_bytes(8, 'little')
 720 | 
 721 | #shortcut for computing the txid because blk files don't store txids
 722 | def compute_txid(raw_tx_bytes: bytes) -> bytes:
 723 |     
 724 |     stream = BytesIO(raw_tx_bytes)
 725 | 
 726 |     # Read version
 727 |     version = stream.read(4)
 728 | 
 729 |     # Peek at marker/flag to detect SegWit
 730 |     marker = stream.read(1)
 731 |     flag = stream.read(1)
 732 |     is_segwit = (marker == b'\x00' and flag == b'\x01')
 733 | 
 734 |     if not is_segwit:
 735 |         # Legacy tx: rewind and hash full raw tx
 736 |         stream.seek(0)
 737 |         stripped_tx = stream.read()
 738 |     else:
 739 |         # Start stripped tx with version
 740 |         stripped_tx = bytearray()
 741 |         stripped_tx += version
 742 | 
 743 |         # Inputs
 744 |         input_count = read_varint(stream)
 745 |         stripped_tx += encode_varint(input_count)
 746 |         for _ in range(input_count):
 747 |             stripped_tx += stream.read(32)  # prev txid
 748 |             stripped_tx += stream.read(4)   # vout index
 749 |             script_len = read_varint(stream)
 750 |             stripped_tx += encode_varint(script_len)
 751 |             stripped_tx += stream.read(script_len)
 752 |             stripped_tx += stream.read(4)   # sequence
 753 | 
 754 |         # Outputs
 755 |         output_count = read_varint(stream)
 756 |         stripped_tx += encode_varint(output_count)
 757 |         for _ in range(output_count):
 758 |             stripped_tx += stream.read(8)   # value
 759 |             script_len = read_varint(stream)
 760 |             stripped_tx += encode_varint(script_len)
 761 |             stripped_tx += stream.read(script_len)
 762 | 
 763 |         # Skip witness data
 764 |         for _ in range(input_count):
 765 |             stack_count = read_varint(stream)
 766 |             for _ in range(stack_count):
 767 |                 item_len = read_varint(stream)
 768 |                 stream.read(item_len)
 769 | 
 770 |         # Locktime
 771 |         stripped_tx += stream.read(4)
 772 | 
 773 |     return hashlib.sha256(hashlib.sha256(stripped_tx).digest()).digest()[::-1]
 774 | 
 775 | 
 776 | # read in all blocks needed
 777 | for bh in block_hashes_needed:
 778 |     block_num += 1
 779 |     print_progress = block_num / len(block_hashes_needed) * 100
 780 |     if print_progress > print_next and print_next < 100:
 781 |         #print(f"{int(print_next)}% ", end="", flush=True)
 782 |         print(f"{int(print_next)}%..",end="",flush=True)
 783 |         print_next += 1
 784 |         if print_next % 7 == 0:
 785 |             print("\n", end="")
 786 | 
 787 |     # Get raw block hex using RPC
 788 |     raw_block_hex = Ask_Node(["getblock", bh, 0], False).decode().strip()
 789 |     raw_block_bytes = binascii.unhexlify(raw_block_hex)
 790 |     stream = BytesIO(raw_block_bytes)
 791 | 
 792 |     # Read header (skip 80 bytes)
 793 |     stream.read(80)
 794 |     tx_count = read_varint(stream)
 795 |     
 796 |     # loop through all txs in this block
 797 |     txs_to_add = []
 798 |     for tx_index in range(tx_count):
 799 |         start_tx = stream.tell()
 800 |         version = stream.read(4)
 801 | 
 802 |         # Check for SegWit
 803 |         marker_flag = stream.read(2)
 804 |         is_segwit = marker_flag == b'\x00\x01'
 805 |         if not is_segwit:
 806 |             stream.seek(start_tx + 4)
 807 | 
 808 |         # Read inputs
 809 |         input_count = read_varint(stream)
 810 |         inputs = []
 811 |         has_op_return = False
 812 |         witness_exceeds = False
 813 |         is_coinbase = False
 814 |         input_txids = []
 815 |         for _ in range(input_count):
 816 |             prev_txid = stream.read(32)
 817 |             prev_index = stream.read(4)
 818 |             script_len = read_varint(stream)
 819 |             script = stream.read(script_len)
 820 |             stream.read(4)
 821 |             input_txids.append(prev_txid[::-1].hex())
 822 |             if prev_txid == b'\x00' * 32 and prev_index == b'\xff\xff\xff\xff':
 823 |                 is_coinbase = True
 824 |             inputs.append({"script": script})
 825 | 
 826 |         #read outputs
 827 |         output_count = read_varint(stream)
 828 |         output_values = []
 829 |         for _ in range(output_count):
 830 |             value_sats = unpack("<Q", stream.read(8))[0]
 831 |             script_len = read_varint(stream)
 832 |             script = stream.read(script_len)
 833 |             if script and script[0] == 0x6a:
 834 |                 has_op_return = True
 835 |             value_btc = value_sats / 1e8
 836 |             if 1e-5 < value_btc < 1e5:
 837 |                 output_values.append(value_btc)
 838 | 
 839 |         # check witness data
 840 |         if is_segwit:
 841 |             for input_data in inputs:
 842 |                 stack_count = read_varint(stream)
 843 |                 total_witness_len = 0
 844 |                 for _ in range(stack_count):
 845 |                     item_len = read_varint(stream)
 846 |                     total_witness_len += item_len
 847 |                     stream.read(item_len)
 848 |                     if item_len > 500 or total_witness_len > 500:
 849 |                         witness_exceeds = True
 850 | 
 851 |         #comput txid
 852 |         stream.read(4)
 853 |         end_tx = stream.tell()
 854 |         stream.seek(start_tx)
 855 |         raw_tx = stream.read(end_tx - start_tx)
 856 |         txid = compute_txid(raw_tx)
 857 |         todays_txids.add(txid.hex())
 858 | 
 859 |         #check same day tx
 860 |         is_same_day_tx = any(itxid in todays_txids for itxid in input_txids)
 861 | 
 862 |         # apply filter and add output to bell curve
 863 |         if (input_count <= 5 and output_count == 2 and not is_coinbase and
 864 |             not has_op_return and not witness_exceeds and not is_same_day_tx):
 865 |             for amount in output_values:
 866 |                 amount_log = log10(amount)
 867 |                 percent_in_range = (amount_log - first_bin_value) / range_bin_values
 868 |                 bin_number_est = int(percent_in_range * number_of_bins)
 869 |                 while output_histogram_bins[bin_number_est] <= amount:
 870 |                     bin_number_est += 1
 871 |                 bin_number = bin_number_est - 1
 872 |                 output_histogram_bin_counts[bin_number] += 1.0
 873 |                 txs_to_add.append(amount)
 874 | 
 875 |     # add outputs to raw outputs
 876 |     if len(txs_to_add) > 0:
 877 |         bkh = block_nums_needed[block_num - 1]
 878 |         tm = block_times_needed[block_num - 1]
 879 |         for amt in txs_to_add:
 880 |             raw_outputs.append(amt)
 881 |             block_heights_dec.append(bkh)
 882 |             block_times_dec.append(tm)
 883 | 
 884 | 
 885 | print("100%",flush=True)
 886 | print("\t\t\t\t\t\t95% done",flush=True)
 887 | 
 888 |             
 889 | 
 890 | ##############################################################################
 891 | 
 892 | #  Step 7 - Remove Round Bitcoin Amounts
 893 | 
 894 | ##############################################################################
 895 | 
 896 | # Although most transaction amounts are related to fiat purchasing power, spending 
 897 | # round Bitcoin amounts is still common. This makes it difficult to determine whether 
 898 | # a histogram interval is truly capturing a round fiat value or simply reflecting a round 
 899 | # BTC amount.
 900 | 
 901 | # We can't completely remove round BTC amounts, because when the fiat price of 
 902 | # Bitcoin is near a round number, round BTC and round fiat values can overlap. 
 903 | # Instead of removing these intervals, we smooth them by averaging the values of 
 904 | # the histogram bins directly above and below the round BTC amounts.
 905 | 
 906 | # After smoothing, we normalize the histogram by dividing each bin by the total sum. 
 907 | # This converts the histogram into percentages rather than raw counts. Percentages 
 908 | # are more stable across days with unusually high or low transaction volume, making 
 909 | # the resulting signal more consistent.
 910 | 
 911 | 
 912 | 
 913 | # print update
 914 | print("\nFinding prices and rendering plot",flush=True)
 915 | print("0%..",end="",flush=True)
 916 | 
 917 | #remove outputs below 10k sat (increased from 1k sat in v6)
 918 | for n in range(0,201):
 919 |     output_histogram_bin_counts[n]=0
 920 | 
 921 | #remove outputs above ten btc
 922 | for n in range(1601,len(output_histogram_bin_counts)):
 923 |     output_histogram_bin_counts[n]=0
 924 | 
 925 | #create a list of round btc bin numbers
 926 | round_btc_bins = [
 927 | 201,  # 1k sats
 928 | 401,  # 10k 
 929 | 461,  # 20k
 930 | 496,  # 30k
 931 | 540,  # 50k
 932 | 601,  # 100k 
 933 | 661,  # 200k
 934 | 696,  # 300k
 935 | 740,  # 500k
 936 | 801,  # 0.01 btc
 937 | 861,  # 0.02
 938 | 896,  # 0.03
 939 | 940,  # 0.04
 940 | 1001, # 0.1 
 941 | 1061, # 0.2
 942 | 1096, # 0.3
 943 | 1140, # 0.5
 944 | 1201  # 1 btc
 945 | ]
 946 | 
 947 | #smooth over the round btc amounts
 948 | for r in round_btc_bins:
 949 |     amount_above = output_histogram_bin_counts[r+1]
 950 |     amount_below = output_histogram_bin_counts[r-1]
 951 |     output_histogram_bin_counts[r] = .5*(amount_above+amount_below)
 952 | 
 953 | #get the sum of the curve
 954 | curve_sum = 0.0
 955 | for n in range(201,1601):
 956 |     curve_sum += output_histogram_bin_counts[n]
 957 | 
 958 | #normalize the curve by dividing by it's sum and removing extreme values
 959 | for n in range(201,1601):
 960 |     output_histogram_bin_counts[n] /= curve_sum
 961 |     
 962 |     #remove extremes (0.008 chosen by historical testing)
 963 |     if output_histogram_bin_counts[n] > 0.008:
 964 |         output_histogram_bin_counts[n] = 0.008
 965 | 
 966 | #print update    
 967 | print("20%..",end="",flush=True)
 968 | 
 969 | 
 970 | 
 971 | 
 972 | ##############################################################################
 973 | 
 974 | #  Step 8 - Construct the Price Finding Stencil
 975 | 
 976 | ##############################################################################
 977 | 
 978 | # Users don’t just send round fiat amounts, they also tend to send them in 
 979 | # predictable proportions depending on the amount. For example, users spend $100 
 980 | # far more often than they send $10,000. We use this proportionality both to lock 
 981 | # onto round fiat amounts and to determine which amount is which.
 982 | 
 983 | # Through historical testing, we’ve measured the average histogram bin counts for 
 984 | # each common round fiat amount and hard-code these values into what we call a 
 985 | # price-finding stencil. The most common and heavily weighted amount in the stencil 
 986 | # is $100. Less frequently used fiat amounts are assigned lower weights, since we 
 987 | # expect to see them transacted less often.
 988 | 
 989 | # Even when users are not transacting at a perfectly round fiat amount, their 
 990 | # behavior is still influenced by Bitcoin’s long-term rise in purchasing power. For 
 991 | # example, if we plot a bell curve of output amounts, we find that the center of the 
 992 | # curve shifts toward smaller BTC amounts over time.
 993 | 
 994 | # This provides additional information we can use. We first apply a smooth stencil to 
 995 | # estimate a center of gravity for the price within a broad 20% range. Then, we apply 
 996 | # a spike stencil based on perfectly round USD amounts to refine the price estimate 
 997 | # to within about 0.5%.
 998 | 
 999 | 
1000 | # Parameters
1001 | num_elements = 803
1002 | mean = 411 #(num_elements - 1) / 2  # Center of the array
1003 | std_dev = 201
1004 | 
1005 | #contstruct the smooth stencil
1006 | smooth_stencil = []
1007 | for x in range(num_elements):
1008 |     exp_part = -((x - mean) ** 2) / (2 * (std_dev ** 2))
1009 |     smooth_stencil.append( (.00150 * 2.718281828459045 ** exp_part) + (.0000005 * x) )
1010 | 
1011 | # Load the spike stencil that fine tunes the alignment on popular usd amounts
1012 | 
1013 | spike_stencil = []
1014 | for n in range(0,803):
1015 |     spike_stencil.append(0.0)
1016 |     
1017 | #round usd bin location   #popularity    #usd amount  
1018 | spike_stencil[40] = 0.001300198324984352  # $1
1019 | spike_stencil[141]= 0.001676746949820743  # $5
1020 | spike_stencil[201]= 0.003468805546942046  # $10
1021 | spike_stencil[202]= 0.001991977522512513  # 
1022 | spike_stencil[236]= 0.001905066647961839  # $15
1023 | spike_stencil[261]= 0.003341772718156079  # $20
1024 | spike_stencil[262]= 0.002588902624584287  # 
1025 | spike_stencil[296]= 0.002577893841190244  # $30
1026 | spike_stencil[297]= 0.002733728814200412  # 
1027 | spike_stencil[340]= 0.003076117748975647  # $50
1028 | spike_stencil[341]= 0.005613067550103145  # 
1029 | spike_stencil[342]= 0.003088253178535568  # 
1030 | spike_stencil[400]= 0.002918457489366139  # $100
1031 | spike_stencil[401]= 0.006174500465286022  # 
1032 | spike_stencil[402]= 0.004417068070043504  # 
1033 | spike_stencil[403]= 0.002628663628020371  # 
1034 | spike_stencil[436]= 0.002858828161543839  # $150
1035 | spike_stencil[461]= 0.004097463611984264  # $200
1036 | spike_stencil[462]= 0.003345917406120509  # 
1037 | spike_stencil[496]= 0.002521467726855856  # $300
1038 | spike_stencil[497]= 0.002784125730361008  # 
1039 | spike_stencil[541]= 0.003792850444811335  # $500
1040 | spike_stencil[601]= 0.003688240815848247  # $1000
1041 | spike_stencil[602]= 0.002392400117402263  # 
1042 | spike_stencil[636]= 0.001280993059008106  # $1500
1043 | spike_stencil[661]= 0.001654665137536031  # $2000
1044 | spike_stencil[662]= 0.001395501347054946  # 
1045 | spike_stencil[741]= 0.001154279140906312  # $5000
1046 | spike_stencil[801]= 0.000832244504868709  # $10000
1047 | 
1048 | 
1049 | 
1050 | ##############################################################################
1051 | 
1052 | #  Step 9 - Estimate a Rough Price
1053 | 
1054 | ##############################################################################
1055 | 
1056 | # To find where the smooth and spiked stencils fit best over the output histogram, we 
1057 | # slide the stencils across the histogram and calculate a score at each position. 
1058 | # There are several ways to do this, but the method we have found most effective is 
1059 | # to multiply the stencil heights by the histogram values at each point and sum the 
1060 | # result.
1061 | 
1062 | # We define upper and lower bounds for the slide range, which set the maximum and 
1063 | # minimum possible prices. These limits will need to be updated in future versions of 
1064 | # UTXOracle as Bitcoin’s purchasing power changes. We also assign a relative 
1065 | # weight to the smooth stencil compared to the spiked stencil. Historical testing has 
1066 | # shown that a weighting ratio of 0.65 to 1 gives the best results.
1067 | 
1068 | # Once the stencils have been slid across the full range, we identify the position with 
1069 | # the highest score. This gives us a rough estimate of the price, accurate to within 
1070 | # about 0.5 percent.
1071 | 
1072 | 
1073 | 
1074 | # set up scores for sliding the stencil
1075 | best_slide        = 0
1076 | best_slide_score  = 0
1077 | total_score       = 0
1078 | 
1079 | #weighting of the smooth and spike slide scores
1080 | smooth_weight     = 0.65
1081 | spike_weight      = 1
1082 | 
1083 | #establish the center slide such that if zero slide then 0.001 btc is $100 ($100k price)
1084 | center_p001 = 601   # 601 is where 0.001 btc is in the output bell curve
1085 | left_p001   = center_p001 - int((len(spike_stencil) +1)/2)
1086 | right_p001  = center_p001 + int((len(spike_stencil) +1)/2)
1087 | 
1088 | #upper and lower limits for sliding the stencil
1089 | min_slide = -141   # $500k
1090 | max_slide =  201   # $5k
1091 |     
1092 | #slide the stencil and calculate slide score
1093 | for slide in range(min_slide,max_slide):
1094 |     
1095 |     #shift the bell curve by the slide
1096 |     shifted_curve = output_histogram_bin_counts[left_p001+slide:right_p001+slide]
1097 |     
1098 |     #score the smoothslide by multiplying the curve by the stencil
1099 |     slide_score_smooth = 0.0
1100 |     for n in range(0,len(smooth_stencil)):
1101 |         slide_score_smooth += shifted_curve[n]*smooth_stencil[n]
1102 |     
1103 |     #score the spiky slide by multiplying the curve by the stencil
1104 |     slide_score = 0.0
1105 |     for n in range(0,len(spike_stencil)):
1106 |         slide_score += shifted_curve[n]*spike_stencil[n]
1107 |     
1108 |     # add the spike and smooth slide scores, neglect smooth slide over wrong regions
1109 |     if slide < 150:
1110 |         slide_score = slide_score + slide_score_smooth*.65
1111 |         
1112 |     # see if this score is the best so far
1113 |     if slide_score > best_slide_score:
1114 |         best_slide_score = slide_score
1115 |         best_slide = slide
1116 |     
1117 |     # increment the total score
1118 |     total_score += slide_score
1119 |         
1120 | # estimate the usd price of the best slide
1121 | usd100_in_btc_best = output_histogram_bins[center_p001+best_slide]
1122 | btc_in_usd_best = 100/(usd100_in_btc_best)
1123 | 
1124 | #find best slide neighbor up
1125 | neighbor_up = output_histogram_bin_counts[left_p001+best_slide+1:right_p001+best_slide+1]
1126 | neighbor_up_score = 0.0
1127 | for n in range(0,len(spike_stencil)):
1128 |     neighbor_up_score += neighbor_up[n]*spike_stencil[n]
1129 | 
1130 | #find best slide neighbor down
1131 | neighbor_down = output_histogram_bin_counts[left_p001+best_slide-1:right_p001+best_slide-1]
1132 | neighbor_down_score = 0.0
1133 | for n in range(0,len(spike_stencil)):
1134 |     neighbor_down_score += neighbor_down[n]*spike_stencil[n]
1135 | 
1136 | #get best neighbor
1137 | best_neighbor = +1
1138 | neighbor_score = neighbor_up_score
1139 | if neighbor_down_score > neighbor_up_score:
1140 |     best_neighbor = -1
1141 |     neighbor_score = neighbor_down_score
1142 | 
1143 | #get best neighbor usd price
1144 | usd100_in_btc_2nd = output_histogram_bins[center_p001+best_slide+best_neighbor]
1145 | btc_in_usd_2nd = 100/(usd100_in_btc_2nd)
1146 | 
1147 | #weight average the two usd price estimates
1148 | avg_score = total_score/len(range(min_slide,max_slide))
1149 | a1 = best_slide_score - avg_score
1150 | a2 = abs(neighbor_score - avg_score)
1151 | w1 = a1/(a1+a2)
1152 | w2 = a2/(a1+a2)
1153 | rough_price_estimate = int(w1*btc_in_usd_best + w2*btc_in_usd_2nd)
1154 | 
1155 | # Print update
1156 | print("40%..",end="",flush=True)
1157 | 
1158 | 
1159 | 
1160 | 
1161 | ##############################################################################
1162 | 
1163 | #  Step 10 - Create Intraday Price Points
1164 | 
1165 | ##############################################################################
1166 | 
1167 | # Using the rough price estimate and the known common fiat spending amounts, we 
1168 | # create a window and assign a rough fiat price to every Bitcoin output. The rough 
1169 | # price does not need to be exact, because the center of mass within this window 
1170 | # shows how the estimate needs to be refined in order to calculate the precise 
1171 | # average.
1172 | 
1173 | # We set the maximum allowed range of price refinement to 25 percent in either 
1174 | # direction. This range accounts for possible errors in the rough estimate, especially 
1175 | # on highly volatile fiat price days. Note that this does not mean UTXOracle can only 
1176 | # handle 25 percent daily price swings. The rough price estimate already captures 
1177 | # most of the daily volatility, so the remaining error is usually much smaller.
1178 | 
1179 | # As we did when finding the rough estimate, we need to remove round BTC 
1180 | # amounts before refining. However, since we are not inserting outputs into a 
1181 | # histogram at this stage, we cannot simply smooth over histogram bins. Instead, we 
1182 | # identify and exclude round BTC amounts by checking whether an output falls within 
1183 | # a narrow range around common round BTC values.
1184 | 
1185 | 
1186 | 
1187 | # list of round USD prices to collect outputs from
1188 | usds = [5,10,15,20,25,30,40,50,100,150,200,300,500,1000]
1189 | 
1190 | # pct of price increase of decrease to include
1191 | pct_range_wide = .25
1192 | 
1193 | # filter for micro round satoshi amounts
1194 | micro_remove_list = []
1195 | i = .00005000
1196 | while i<.0001:
1197 |     micro_remove_list.append(i)
1198 |     i += .00001
1199 | i = .0001
1200 | while i<.001:
1201 |     micro_remove_list.append(i)
1202 |     i += .00001
1203 | i = .001
1204 | while i<.01:
1205 |     micro_remove_list.append(i)
1206 |     i += .0001
1207 | i = .01
1208 | while i<.1:
1209 |     micro_remove_list.append(i)
1210 |     i += .001
1211 | i = .1
1212 | while i<1:
1213 |     micro_remove_list.append(i)
1214 |     i += .01
1215 | pct_micro_remove = .0001
1216 | 
1217 | # init output prices list
1218 | output_prices = []
1219 | output_blocks = []
1220 | output_times = []
1221 | 
1222 | #loop through all outputs
1223 | for i in range (0,len(raw_outputs)):
1224 |     
1225 |     #get the amount, height and time of the next output
1226 |     n = raw_outputs[i]
1227 |     b = block_heights_dec[i]
1228 |     t = block_times_dec[i]
1229 |     
1230 |     #loop throughll usd amounts possible
1231 |     for usd in usds:
1232 |         
1233 |         #calculate the upper and lower bounds for the USD range
1234 |         avbtc = usd/rough_price_estimate
1235 |         btc_up = avbtc + pct_range_wide * avbtc 
1236 |         btc_dn = avbtc - pct_range_wide * avbtc
1237 |     
1238 |         # check if inside price bounds
1239 |         if btc_dn < n < btc_up:
1240 |             append = True
1241 |             
1242 |             #remove perfectly round sats
1243 |             for r in micro_remove_list:
1244 |                 
1245 |                 rm_dn = r - pct_micro_remove * r
1246 |                 rm_up = r + pct_micro_remove * r
1247 |                 if rm_dn < n < rm_up:
1248 |                     append = False
1249 |             
1250 |             # if in price range and not perfectly round sat, add to list
1251 |             if append:
1252 |                 output_prices.append(usd/n)
1253 |                 output_blocks.append(b)
1254 |                 output_times.append(t)
1255 | 
1256 | print("60%..",end="",flush=True)
1257 | 
1258 | 
1259 | 
1260 | 
1261 | ##############################################################################
1262 | 
1263 | #  Step 11 - Find the Exact Average Price
1264 | 
1265 | ##############################################################################
1266 | 
1267 | # We find the exact average price using an iterative procedure that locates the 
1268 | # central output within the cluster of rough intraday price points. The algorithm 
1269 | # calculates the center of mass of the intraday price points and shifts the price 
1270 | # estimate toward this center.
1271 | 
1272 | # As the window shifts, price points drop out on one side and new ones are added on 
1273 | # the other. This changes the distribution, so the center of mass must be recalculated 
1274 | # each time. By repeating this process, one of two outcomes will occur: either the 
1275 | # center price will converge to a single stable value, or it will begin to oscillate in a 
1276 | # repeating pattern, shifting up and down by fixed amounts.
1277 | 
1278 | # The final exact price is determined by one of two conditions: A) the converged 
1279 | # center price, or B) the first central value where stable oscillation begins.
1280 | 
1281 | 
1282 | # define an algorithm for finding the central price point and avg deviation
1283 | def find_central_output(r2, price_min, price_max):
1284 |     
1285 |     #sort the list of prices
1286 |     r6 = [r for r in r2 if price_min < r < price_max]
1287 |     outputs = sorted(r6)
1288 |     n = len(outputs)
1289 | 
1290 |     # Prefix sums
1291 |     prefix_sum = []
1292 |     total = 0
1293 |     for x in outputs:
1294 |         total += x
1295 |         prefix_sum.append(total)
1296 | 
1297 |     # count the number of point left and right
1298 |     left_counts = list(range(n))
1299 |     right_counts = [n - i - 1 for i in left_counts]
1300 |     left_sums = [0] + prefix_sum[:-1]
1301 |     right_sums = [total - x for x in prefix_sum]
1302 | 
1303 |     # find the total distance to other points
1304 |     total_dists = []
1305 |     for i in range(n):
1306 |         dist = (outputs[i] * left_counts[i] - left_sums[i]) + (right_sums[i] - outputs[i] * right_counts[i])
1307 |         total_dists.append(dist)
1308 | 
1309 |     # find the Most central output
1310 |     min_index, _ = min(enumerate(total_dists), key=lambda x: x[1])
1311 |     best_output = outputs[min_index]
1312 | 
1313 |     # Median absolute deviation
1314 |     deviations = [abs(x - best_output) for x in outputs]
1315 |     deviations.sort()
1316 |     m = len(deviations)
1317 |     if m % 2 == 0:
1318 |         mad = (deviations[m//2 - 1] + deviations[m//2]) / 2
1319 |     else:
1320 |         mad = deviations[m//2]
1321 | 
1322 |     return best_output, mad
1323 | 
1324 | 
1325 | # use a tight pct range to find the first central price
1326 | pct_range_tight = .05
1327 | price_up = rough_price_estimate + pct_range_tight * rough_price_estimate 
1328 | price_dn = rough_price_estimate - pct_range_tight * rough_price_estimate
1329 | central_price, av_dev = find_central_output(output_prices,price_dn,price_up)
1330 | 
1331 | # find the deviation as a percentage of the price range
1332 | price_range = price_up - price_dn
1333 | dev_pct = av_dev/price_range
1334 | 
1335 | # iteratively re-center the bounds and find a new center price until convergence
1336 | avs = set()
1337 | avs.add(central_price)
1338 | while central_price not in avs:
1339 |     avs.add(central_price)
1340 |     price_up = central_price + pct_range_tight * central_price 
1341 |     price_dn = central_price - pct_range_tight * central_price
1342 |     central_price, av_dev = find_central_output(output_prices,price_dn,price_up)
1343 |     price_range = price_up - price_dn
1344 |     dev_pct = av_dev/price_range
1345 | 
1346 | #print update
1347 | print("80%..",end="",flush=True)
1348 | 
1349 | #because price flutucation may exceed bounds, check a wide ranger for the devation
1350 | pct_range_med = .1
1351 | price_up = central_price + pct_range_med * central_price 
1352 | price_dn = central_price - pct_range_med * central_price
1353 | price_range = price_up - price_dn
1354 | unused_price, av_dev = find_central_output(output_prices,price_dn,price_up)
1355 | dev_pct = av_dev/price_range
1356 | 
1357 | # use the pct deviation of data to set y axis range
1358 | map_dev_axr = (.15-.05)/(.20-.17)
1359 | ax_range = .05+ (dev_pct-.17)*map_dev_axr
1360 | 
1361 | #set max and min y axis ranges
1362 | if ax_range < .05:
1363 |     ax_range = .05
1364 | if ax_range > .2:
1365 |     ax_range = .2
1366 | price_up = central_price + ax_range * central_price 
1367 | price_dn = central_price - ax_range * central_price
1368 | 
1369 | 
1370 | # print update
1371 | print("100%\t\t\tdone",flush=True)
1372 | if date_mode:
1373 |     print("\n\n\t\t"+price_day_date_utc+" price: $"+f"{int(central_price):,}\n\n",flush=True)
1374 | 
1375 | 
1376 | 
1377 | 
1378 | ##############################################################################
1379 | 
1380 | # Step 12 - Generate a Price Plot HTML Page
1381 | 
1382 | ##############################################################################
1383 | 
1384 | # To display the results, we create a local webpage and serve an intraday plot using 
1385 | # an HTML <canvas> element rendered in the user's browser. This is accomplished 
1386 | # by generating a string that follows HTML and JavaScript syntax, defining the 
1387 | # canvas and drawing the intraday price points as (x, y) elements.
1388 | 
1389 | # The resulting string is saved as a local HTML file, which should automatically open 
1390 | # in the user’s default web browser using the webbrowser.open() command.
1391 | 
1392 | 
1393 | # geometric layout of webpage and chart
1394 | width = 1000   # canvas width (px)
1395 | height = 660  # canvas height (px)
1396 | margin_left = 120
1397 | margin_right = 90
1398 | margin_top = 100
1399 | margin_bottom = 120
1400 | 
1401 | # #get linear block hieghts on x axis
1402 | start = block_nums_needed[0]
1403 | end = block_nums_needed[-1]
1404 | count = len(output_prices)
1405 | step = (end - start) / (count - 1) if count > 1 else 0
1406 | b3 = [start + i * step for i in range(count)]
1407 | 
1408 | #Prepare python prices, heights, and timestamps for sending to html
1409 | heights = []
1410 | heights_smooth = []
1411 | timestamps = []
1412 | prices = []
1413 | for i in range(len(output_prices)):
1414 |     if price_dn < output_prices[i] < price_up:
1415 |         heights.append(output_blocks[i])
1416 |         heights_smooth.append(b3[i])
1417 |         timestamps.append(output_times[i])
1418 |         prices.append(output_prices[i])
1419 | 
1420 | # Sort by timestamps (optional, looks nicer)
1421 | heights_smooth, prices = zip(*sorted(zip(heights_smooth, prices)))
1422 | 
1423 | # set x tick locations
1424 | num_ticks = 5
1425 | n = len(heights_smooth)
1426 | tick_indxs = [round(i * (n - 1) / (num_ticks - 1)) for i in range(num_ticks)]
1427 | 
1428 | # Set the x tick labels 
1429 | xtick_positions = []
1430 | xtick_labels = []
1431 | for tk in tick_indxs:
1432 |     xtick_positions.append(heights_smooth[tk])
1433 |     block_height = heights[tk]
1434 |     timestamp = timestamps[tk]
1435 |     dt = datetime.utcfromtimestamp(timestamp)
1436 |     time_label = f"{dt.hour:02}:{dt.minute:02} UTC"
1437 |     label = f"{block_height}\n{time_label}"  # comma after 3 digits
1438 |     xtick_labels.append(label)
1439 | 
1440 | # Calculate avg price
1441 | avg_price = central_price
1442 | 
1443 | #set the plot annotations
1444 | plot_title_left = ""
1445 | plot_title_right = ""
1446 | bottom_note1 = ""
1447 | bottom_note2 = ""
1448 | if date_mode:
1449 |     plot_title_left = price_day_date_utc+" blocks from local node"
1450 |     plot_title_right ="UTXOracle Consensus Price $"+f"{int(central_price):,} "#+test_price
1451 |     bottom_note1 = "Consensus Data:"
1452 |     bottom_note2 = "this plot is identical and immutable for every bitcoin node"
1453 | if block_mode:
1454 |     plot_title_left = "Local Node Blocks "+str(block_start_num)+"-"+str(block_finish_num)
1455 |     plot_title_right ="UTXOracle Block Window Price $"+f"{int(central_price):,}"
1456 |     bottom_note1 = "* Block Window Price "
1457 |     bottom_note2 = "may have node dependent differences data on the chain tip"
1458 | 
1459 | 
1460 | # Write the HTML code for the chart
1461 | html_content = f'''<!DOCTYPE html>
1462 | 
1463 | <html>
1464 | <head>
1465 | <title>UTXOracle Local</title>
1466 | <style>
1467 |     body {{
1468 |         background-color: black;
1469 |         margin: 0;
1470 |         color: #CCCCCC;
1471 |         font-family: Arial, sans-serif;
1472 |         text-align: center;
1473 |     }}
1474 |     canvas {{
1475 |         background-color: black;
1476 |         display: block;
1477 |         margin: auto;
1478 |     }}
1479 |     
1480 |     
1481 | </style>
1482 | </head>
1483 | <body>
1484 | 
1485 | 
1486 | 
1487 | <div id="tooltip" style="
1488 |     position: absolute;
1489 |     background-color: black;
1490 |     color: cyan;
1491 |     border: 1px solid cyan;
1492 |     padding: 8px;
1493 |     font-size: 14px;
1494 |     border-radius: 5px;
1495 |     pointer-events: none;
1496 |     opacity: 0;
1497 |     transition: opacity 0.2s;
1498 |     text-align: left;
1499 |     z-index: 10;
1500 | "></div>
1501 | 
1502 | <div style="position: relative; width: 95%; max-width: 1000px; margin: auto;">
1503 |     
1504 |     <canvas id="myCanvas" style="width: 100%; height: auto;" width="{width}" height="{height}"></canvas>
1505 |     
1506 |     <button id="downloadBtn" style="
1507 |         position: absolute;
1508 |         bottom: 5%;
1509 |         right: 2%;
1510 |         font-size: 14px;
1511 |         padding: 6px 10px;
1512 |         background-color: black;
1513 |         color: white;
1514 |         border: none;
1515 |         border-radius: 5px;
1516 |         cursor: pointer;
1517 |         opacity: 0.85;
1518 |     ">
1519 |     Save PNG
1520 |     </button>
1521 | 
1522 | </div>
1523 | 
1524 | 
1525 | 
1526 | <script>
1527 | const canvas = document.getElementById('myCanvas');
1528 | const ctx = canvas.getContext('2d');
1529 | 
1530 | const width = {width};
1531 | const height = {height};
1532 | 
1533 | const marginLeft = {margin_left};
1534 | const marginRight = {margin_right};
1535 | const marginTop = {margin_top};
1536 | const marginBottom = {margin_bottom};
1537 | 
1538 | const plotWidth = width - marginLeft - marginRight;
1539 | const plotHeight = height - marginTop - marginBottom;
1540 | 
1541 | // Data
1542 | const heights_smooth = {list(heights_smooth)};
1543 | const prices = {list(prices)};
1544 | const heights = {list(heights)};
1545 | const timestamps = {list(timestamps)};
1546 | 
1547 | 
1548 | const ymin = Math.min(...prices);
1549 | const ymax = Math.max(...prices);
1550 | const xmin = Math.min(...heights_smooth);
1551 | const xmax = Math.max(...heights_smooth);
1552 | 
1553 | // xticks
1554 | const xtick_positions = {xtick_positions};
1555 | const xtick_labels = {xtick_labels};
1556 | 
1557 | 
1558 | // Scaling functions
1559 | function scaleX(t) {{
1560 |     return marginLeft + (t - xmin) / (xmax - xmin) * plotWidth;
1561 | }}
1562 | 
1563 | function scaleY(p) {{
1564 |     return marginTop + (1 - (p - ymin) / (ymax - ymin)) * plotHeight;
1565 | }}
1566 | 
1567 | // Background
1568 | ctx.fillStyle = "black";
1569 | ctx.fillRect(0, 0, width, height);
1570 | 
1571 | 
1572 | // UTXOracle Local Title
1573 | ctx.font = "bold 36px Arial";
1574 | ctx.textAlign = "center";
1575 | ctx.fillStyle = "cyan";
1576 | ctx.fillText("UTXOracle", width / 2 - 60, 40);  // Slight left shift for spacing
1577 | 
1578 | ctx.fillStyle = "lime";
1579 | ctx.fillText("Local", width / 2 + 95, 40);  // Slight right shift
1580 | 
1581 | // Plot Date and Consensus Price
1582 | ctx.font = "24px Arial";
1583 | ctx.textAlign = "right";
1584 | ctx.fillStyle = "white";
1585 | ctx.fillText("{plot_title_left}", width /2, 80);
1586 | ctx.textAlign = "left";
1587 | ctx.fillStyle = "lime";
1588 | ctx.fillText("{plot_title_right}", width/2 +10, 80);
1589 | 
1590 | 
1591 | 
1592 | 
1593 | 
1594 | // Draw axes
1595 | ctx.strokeStyle = "white";
1596 | ctx.lineWidth = 1;
1597 | 
1598 | // Y axis
1599 | ctx.beginPath();
1600 | ctx.moveTo(marginLeft, marginTop);
1601 | ctx.lineTo(marginLeft, marginTop + plotHeight);
1602 | ctx.stroke();
1603 | 
1604 | // X axis
1605 | ctx.beginPath();
1606 | ctx.moveTo(marginLeft, marginTop + plotHeight);
1607 | ctx.lineTo(marginLeft + plotWidth, marginTop + plotHeight);
1608 | ctx.stroke();
1609 | 
1610 | // Right spine
1611 | ctx.beginPath();
1612 | ctx.moveTo(marginLeft + plotWidth, marginTop);
1613 | ctx.lineTo(marginLeft + plotWidth, marginTop + plotHeight);
1614 | ctx.stroke();
1615 | 
1616 | // Top spine
1617 | ctx.beginPath();
1618 | ctx.moveTo(marginLeft, marginTop);
1619 | ctx.lineTo(marginLeft + plotWidth, marginTop);
1620 | ctx.stroke();
1621 | 
1622 | 
1623 | // Draw ticks and labels
1624 | ctx.fillStyle = "white";
1625 | ctx.font = "20px Arial";
1626 | 
1627 | // Y axis ticks
1628 | const yticks = 5;
1629 | for (let i = 0; i <= yticks; i++) {{
1630 |     let p = ymin + (ymax - ymin) * i / yticks;
1631 |     let y = scaleY(p);
1632 |     ctx.beginPath();
1633 |     ctx.moveTo(marginLeft - 5, y);
1634 |     ctx.lineTo(marginLeft, y);
1635 |     ctx.stroke();
1636 |     ctx.textAlign = "right";
1637 |     ctx.fillText(Math.round(p).toLocaleString(), marginLeft - 10, y + 4);
1638 | }}
1639 | 
1640 | // X axis ticks
1641 | ctx.textAlign = "center";
1642 | ctx.font = "16px Arial";
1643 | 
1644 | for (let i = 0; i < xtick_positions.length; i++) {{
1645 |     let x = scaleX(xtick_positions[i]);
1646 |     ctx.beginPath();
1647 |     ctx.moveTo(x, marginTop + plotHeight);
1648 |     ctx.lineTo(x, marginTop + plotHeight + 5);
1649 |     ctx.stroke();
1650 | 
1651 |     // Split label into two lines
1652 |     let parts = xtick_labels[i].split("\\n");
1653 |     ctx.fillText(parts[0], x, marginTop + plotHeight + 20);   // Block height
1654 |     ctx.fillText(parts[1], x, marginTop + plotHeight + 40);   // Time
1655 | }}
1656 | 
1657 | 
1658 | // Axis labels
1659 | ctx.fillStyle = "white";
1660 | ctx.font = "20px Arial";
1661 | ctx.textAlign = "center";
1662 | ctx.fillText("Block Height and UTC Time", marginLeft + plotWidth/2, height - 48);
1663 | ctx.save();
1664 | ctx.translate(20, marginTop + plotHeight/2);
1665 | ctx.rotate(-Math.PI / 2);
1666 | ctx.fillText("BTC Price ($)", 0, 0);
1667 | ctx.restore();
1668 | 
1669 | // Plot points
1670 | ctx.fillStyle = "cyan";
1671 | for (let i = 0; i < heights_smooth.length; i++) {{
1672 |     let x = scaleX(heights_smooth[i]);
1673 |     let y = scaleY(prices[i]);
1674 |     ctx.fillRect(x, y, .75, .75);
1675 | }}
1676 | 
1677 | // Annotation for average price
1678 | ctx.fillStyle = "cyan";
1679 | ctx.font = "20px Arial";
1680 | ctx.textAlign = "left";
1681 | ctx.fillText("- {int(avg_price):,}", marginLeft + plotWidth + 1, scaleY({avg_price}) +0);
1682 | 
1683 | 
1684 | // Annotate bottom chart note
1685 | 
1686 | ctx.font = "24px Arial";
1687 | ctx.fillStyle = "lime";
1688 | ctx.textAlign = "right";
1689 | ctx.fillText("{bottom_note1}", 320, height-10);
1690 | ctx.font = "24px Arial";
1691 | ctx.fillStyle = "white";
1692 | ctx.textAlign = "left";
1693 | ctx.fillText("{bottom_note2}", 325, height-10);
1694 | 
1695 | 
1696 | // === MOUSEOVER INFO ===
1697 | 
1698 | const tooltip = document.getElementById('tooltip');
1699 | 
1700 | canvas.addEventListener('mousemove', function(event) {{
1701 | const rect = canvas.getBoundingClientRect();
1702 | const scaleX = canvas.width / rect.width;
1703 | const scaleY = canvas.height / rect.height;
1704 | const mouseX = (event.clientX - rect.left) * scaleX;
1705 | const mouseY = (event.clientY - rect.top) * scaleY;
1706 | 
1707 | if (mouseX >= marginLeft && mouseX <= width - marginRight &&
1708 |     mouseY >= marginTop && mouseY <= marginTop + plotHeight) {{
1709 | 
1710 |     const fractionAcross = (mouseX - marginLeft) / plotWidth;
1711 |     let index = Math.round(fractionAcross * (heights.length - 1));
1712 |     index = Math.max(0, Math.min(index, heights.length - 1));
1713 | 
1714 |     const price = ymax - (mouseY - marginTop) / plotHeight * (ymax - ymin);
1715 |     const blockHeight = heights[index];
1716 |     const timestamp = timestamps[index];
1717 | 
1718 |     const date = new Date(timestamp * 1000);
1719 |     const hours = date.getUTCHours().toString().padStart(2, '0');
1720 |     const minutes = date.getUTCMinutes().toString().padStart(2, '0');
1721 |     const utcTime = `${{hours}}:${{minutes}} UTC`;
1722 | 
1723 |     tooltip.innerHTML = `
1724 |         Price: $${{Math.round(price).toLocaleString()}}<br>
1725 |         Block: ${{blockHeight.toLocaleString()}}<br>
1726 |         Time: ${{utcTime}}
1727 |     `;
1728 | 
1729 |     tooltip.style.left = (event.clientX + 5) + 'px';
1730 |     tooltip.style.top = (event.clientY + window.scrollY - 75) + 'px';
1731 |     tooltip.style.opacity = 1;
1732 | }} else {{
1733 |     tooltip.style.opacity = 0;
1734 | }}
1735 | }});
1736 | 
1737 | 
1738 | 
1739 | 
1740 | </script>
1741 | 
1742 | 
1743 | <script>
1744 | // Download canvas as PNG
1745 | const downloadBtn = document.getElementById('downloadBtn');
1746 | downloadBtn.addEventListener('click', function() {{
1747 |     const link = document.createElement('a');
1748 |     link.download = 'UTXOracle_Local_Node_Price.png';
1749 |     link.href = canvas.toDataURL('image/png');
1750 |     link.click();
1751 | }});
1752 | </script>
1753 | 
1754 | <br>
1755 | <br>
1756 | 
1757 | 
1758 | <h2 style="margin-top:10px; font-size:24px;">
1759 | Want a 
1760 | <span style="color:orange; font-size:24px;">Live Updating Oracle</span>
1761 |   with New Mempool Transactions? 
1762 | </h2>
1763 | 
1764 | 
1765 | <iframe
1766 |     width="{width}"
1767 |     height="{height}"
1768 |     src="https://www.youtube.com/embed/live_stream?channel=UCXN7Xa_BF7dqLErzOmS-B7Q&autoplay=1&mute=1"
1769 |     frameborder="0"
1770 |     allow="autoplay; encrypted-media"
1771 |     allowfullscreen
1772 |     
1773 | </iframe>
1774 | 
1775 | 
1776 | 
1777 | <br>
1778 | 
1779 | </body>
1780 | </html>
1781 | '''
1782 | 
1783 | 
1784 | # name the file with dates or blocks
1785 | filename = ".html"
1786 | if date_mode:
1787 |     filename = "UTXOracle_"+price_date_dash+filename
1788 | if block_mode:
1789 |     filename = "UTXOracle_"+str(block_start_num)+"-"+str(block_finish_num)+filename
1790 | 
1791 | 
1792 | # Write file locally and serve to browser
1793 | import webbrowser
1794 | with open(filename, "w") as f:
1795 |     f.write(html_content)
1796 | webbrowser.open('file://' + os.path.realpath(filename))
1797 | 
1798 | 
1799 | 
1800 | 
1801 | 


--------------------------------------------------------------------------------