├── README.md ├── check_nest.sh ├── collect.php ├── dbsetup ├── device_info.php ├── fetch.php ├── inc ├── class.db.php └── config.php ├── index.html ├── insert.php ├── nest-api-php-workaround-login.php ├── nestgraph-screenshot.png └── test.php /README.md: -------------------------------------------------------------------------------- 1 | # nestgraph 2 | 3 | Create pretty charts of your Nest thermostat data. 4 | 5 | ## Background 6 | 7 | The point of this project was to see how well the Nest algorithms work. In particuar, the Nest claims to minimize overshoot, which is a common problem with cast-iron radiators. It also claims to know when to start heating in order to hit your target temperature exactly at the time you scheduled it. 8 | 9 | Unfortunately, you can't actually access historical temperature data on the Nest website or via the iOS app. It shows you when heating was turned on/off and what the temperature targets were at those times, but it doesn't give you any indication of how well or how poorly the thermostat performed. This could be by design, as it's a lot of information to store. 10 | 11 | This project uses an unofficial Nest API to pull your temperature readings periodically and store them in a database so that you can inspect the data yourself in an easily consumable form. 12 | 13 | I also wanted an excuse to play with the [D3](http://d3js.org) (Data-Driven Documents) library a little. 14 | 15 | ## Features 16 | 17 | * Polls Nest website to collect thermostat telemetry 18 | * Stores selected data in local MySQL database 19 | * Generates a nice visualization of actual temp vs. set point 20 | * Lower mini-chart is interactive pan-and-zoom of the upper chart 21 | * Hover over the gray circles to get the exact timestamp and temperature 22 | 23 | ![nestgraph screenshot](https://github.com/chriseng/nestgraph/raw/master/nestgraph-screenshot.png) 24 | 25 | ## Dependencies 26 | 27 | * LAMP stack 28 | * Unofficial [nest-api](https://github.com/gboudreau/nest-api) library by Guillaume Boudreau 29 | 30 | ## Getting Started 31 | 32 | Clone this repo into your web root. 33 | 34 | ```bash 35 | cd [your-web-root] 36 | git clone https://github.com/chriseng/nestgraph.git 37 | ``` 38 | 39 | Grab a copy of nest-api and unzip into the ```nestgraph``` directory you created in the previous step. It should create a subdirectory called ```nest-api-master```. 40 | 41 | ```bash 42 | cd nestgraph 43 | wget https://github.com/gboudreau/nest-api/archive/master.zip 44 | unzip master.zip 45 | rm -f master.zip 46 | ``` 47 | Open ```inc/config.php``` in a text editor and update the ```nest_user``` and ```nest_pass``` variables with your username and password for nest.com. Update the ```local_tz``` variable to reflect your time zone. 48 | 49 | As of January 2020, the nest-api library is unable to authenticate directly to the Google Nest API. So instead you have to copy/paste in a session credential which will be cached and used until it expires. At which point you have to do it again. Run the ```nest-api-php-workaround-login.php``` script (copied into this repo from its [original location](https://gist.github.com/gboudreau/8b8851a9c99140b6234856bbc80a2d24)), and follow the instructions. 50 | 51 | ```bash 52 | php nest-api-php-workaround-login.php 53 | ``` 54 | 55 | Once you've done that, run the test script to make sure that the API is able to pull your thermostat data correctly from nest.com. 56 | 57 | ```bash 58 | php test.php 59 | ``` 60 | 61 | If this works, you should see a bunch of stuff fly across the screen, ending with something like this: 62 | 63 | ```bash 64 | Heating : 0 65 | Timestamp : 2013-01-15 22:10:39 66 | Target temperature : 67.00 67 | Current temperature : 67.53 68 | Current humidity : 29 69 | ``` 70 | 71 | Choose a password for your local MySQL nest database, and update it in two places: ```inc/config.php``` (the ```db_pass``` variable) and ```dbsetup```. 72 | 73 | As root or using a DBA account, run the commands in dbsetup to create the MySQL database that will be used to store historical data. 74 | 75 | ```bash 76 | mysql -u root < dbsetup 77 | ``` 78 | 79 | Create a cron job to poll the website periodically and update the local database. The thermostat does not phone home on a fixed schedule, but typically it updates in 5 to 30 minute intervals. The script will only insert into the database if there is new data available. Obviously, update the path to ```insert.php``` if it's not in ```/var/www/html/nestgraph```. 80 | 81 | ```bash 82 | */5 * * * * /usr/bin/php /var/www/html/nestgraph/insert.php > /dev/null 83 | ``` 84 | 85 | Optionally, create a cron job to check periodically if your cached session credential is still valid and if your thermostat has gone offline. I have mine run every 30 minutes; adjust as appropriate. Populate the recipient email(s) in ```check_nest.sh``` if you want email notifications, then add the crontab entry, again updating the below to use the appropriate execution path for your system. 86 | 87 | ```bash 88 | */30 * * * * /var/www/html/nestgraph/check_nest.sh 89 | ``` 90 | 91 | Point web browser to the ```nestgraph``` directory on your webserver! Admire pretty graphs (actually, they won't be all that pretty until it has collected some data). 92 | 93 | 94 | ## Known Issues 95 | 96 | * Only checks for heating on/off, not cooling (I don't have cooling) 97 | * Only supports a single Nest thermostat (I only have one) 98 | * Heating on/off trendline lazily mapped on to the temperature graph 99 | * Assumes you want temperatures displayed in Fahrenheit 100 | * Doesn't automatically redraw when you resize the browser window 101 | * Labels (current/target/heating) don't follow the trend lines when you pan/zoom 102 | * Have to manually update session credential each time it expires due to [Jan 2020 login changes](https://github.com/gboudreau/nest-api/issues/110) 103 | 104 | -------------------------------------------------------------------------------- /check_nest.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # This script will only work if it's running on a host configured to send 4 | # mail using the 'mail' command. It also assumes you have 'grep'. 5 | 6 | # Uncomment and populate if you want to send results to email rather than 7 | # to the console 8 | # RECIPIENTS=('your_email1@example.com' 'your_email2@example.com') 9 | 10 | # Uncomment and populate if you want to override the email sender 11 | # SENDER='your_from_email@example.com' 12 | 13 | if [ -z "${SENDER}" ]; then 14 | MAILFROM="" 15 | else 16 | MAILFROM="-r ${SENDER}" 17 | fi 18 | 19 | function creds_valid() { 20 | if [[ "${1}" == *"invalid user credentials"* ]]; then 21 | false 22 | else 23 | true 24 | fi 25 | } 26 | 27 | function device_online() { 28 | if [[ "${1}" == *"[online] => 1"* ]]; then 29 | true 30 | else 31 | false 32 | fi 33 | } 34 | 35 | function last_connection() { 36 | echo -e "${1}" | grep -o -E "\[last_connection\] => .{19}" | cut -c22- 37 | } 38 | 39 | SCRIPT=$(basename "$0") 40 | DIR=$(cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd) 41 | DEV_INFO=$(php $DIR/device_info.php 2>&1 | sed -e 's/^[[:space:]]*//') 42 | LAST=$(last_connection "${DEV_INFO}") 43 | 44 | if [ -z "${DEV_INFO}" ]; then 45 | exit 1 46 | fi 47 | 48 | if creds_valid "${DEV_INFO}"; then 49 | if ! device_online "${DEV_INFO}"; then 50 | STATUS="Nest device is offline, last seen ${LAST}" 51 | if [[ -n "${RECIPIENTS[@]}" ]]; then 52 | for recip in "${RECIPIENTS[@]}"; do 53 | echo "${STATUS}" | mail ${MAILFROM} -s"${SCRIPT}: device offline" ${recip} 54 | done 55 | else 56 | echo "${STATUS}" 57 | fi 58 | fi 59 | else 60 | STATUS="Nestgraph credential cache has expired" 61 | if [[ -n "${RECIPIENTS[@]}" ]]; then 62 | for recip in "${RECIPIENTS[@]}"; do 63 | echo "${STATUS}" | mail ${MAILFROM} -s"${SCRIPT}: expired session credentials" ${recip} 64 | done 65 | else 66 | echo "${STATUS}" 67 | fi 68 | fi 69 | 70 | -------------------------------------------------------------------------------- /collect.php: -------------------------------------------------------------------------------- 1 | getDeviceInfo(); 14 | $data = array('heating' => ($info->current_state->heat == 1 ? 1 : 0), 15 | 'timestamp' => $info->network->last_connection, 16 | 'target_temp' => sprintf("%.02f", $info->target->temperature), 17 | 'current_temp' => sprintf("%.02f", $info->current_state->temperature), 18 | 'humidity' => $info->current_state->humidity 19 | ); 20 | return $data; 21 | } 22 | 23 | function c_to_f($c) { 24 | return ($c * 1.8) + 32; 25 | } 26 | 27 | ?> -------------------------------------------------------------------------------- /dbsetup: -------------------------------------------------------------------------------- 1 | CREATE DATABASE nest; 2 | GRANT ALL PRIVILEGES ON nest.* TO 'nest_admin'@'localhost' IDENTIFIED BY 'choose_a_db_password'; 3 | FLUSH PRIVILEGES; 4 | 5 | USE nest; 6 | CREATE TABLE `data` ( 7 | `timestamp` timestamp NOT NULL, 8 | `heating` tinyint unsigned NOT NULL, 9 | `target` numeric(7,3) NOT NULL, 10 | `current` numeric(7,3) NOT NULL, 11 | `humidity` tinyint unsigned NOT NULL, 12 | `updated` timestamp NOT NULL, 13 | PRIMARY KEY (`timestamp`), 14 | UNIQUE KEY `timestamp` (`timestamp`) 15 | ) 16 | ENGINE=MyISAM DEFAULT CHARSET=latin1; 17 | 18 | -------------------------------------------------------------------------------- /device_info.php: -------------------------------------------------------------------------------- 1 | getDeviceInfo(); 13 | print_r($infos); 14 | 15 | -------------------------------------------------------------------------------- /fetch.php: -------------------------------------------------------------------------------- 1 | res->prepare("SELECT * from data where timestamp>=DATE_SUB(NOW(), INTERVAL ? HOUR) order by timestamp")) { 16 | $stmt->bind_param("i", $hrs); 17 | $stmt->execute(); 18 | $stmt->bind_result($timestamp, $heating, $target, $current, $humidity, $updated); 19 | header("Content-type: text/tab-separated-values"); 20 | print "timestamp\theating\ttarget\tcurrent\thumidity\tupdated\n"; 21 | while ($stmt->fetch()) { 22 | print implode("\t", array($timestamp, $heating, $target, $current, $humidity, $updated)) . "\n"; 23 | } 24 | $stmt->close(); 25 | } 26 | $db->close(); 27 | } catch (Exception $e) { 28 | $errors[] = ("DB connection error! " . $e->getMessage() . "."); 29 | } 30 | 31 | ?> 32 | -------------------------------------------------------------------------------- /inc/class.db.php: -------------------------------------------------------------------------------- 1 | config = $config; 12 | $this->connect(); 13 | } else { 14 | throw new Exception("DB config information is incomplete."); 15 | } 16 | } 17 | 18 | protected function connect(){ 19 | try { 20 | $this->res = new mysqli($this->config['db_ip'], $this->config['db_user'], $this->config['db_pass'], $this->config['db_name']); 21 | $this->conn = true; 22 | if ($this->res->connect_error){ 23 | $this->conn = false; 24 | throw new Exception("DB error: " . $this->res->connect_error); 25 | } 26 | } catch (Exception $e) { 27 | throw new Exception("DB error: " . $e->getMessage()); 28 | } 29 | } 30 | 31 | public function close() { 32 | try { 33 | $this->res->close(); 34 | $this->conn = false; 35 | } catch (Exception $e) { 36 | throw new Exception("DB error: " . $e->getMessage()); 37 | } 38 | } 39 | } -------------------------------------------------------------------------------- /inc/config.php: -------------------------------------------------------------------------------- 1 | '127.0.0.1', 4 | 'db_user' => 'nest_admin', 5 | 'db_pass' => 'choose_a_db_password', 6 | 'db_name' => 'nest', 7 | 'nest_user' => 'your_nest_username', 8 | 'nest_pass' => 'your_nest_password', 9 | 'local_tz' => 'America/New_York' // see http://php.net/manual/en/timezones.php 10 | ); 11 | 12 | ?> -------------------------------------------------------------------------------- /index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | NestGraph 5 | 31 | 32 | 33 | 34 | 284 | 285 | 286 | -------------------------------------------------------------------------------- /insert.php: -------------------------------------------------------------------------------- 1 | res->prepare("REPLACE INTO data (timestamp, heating, target, current, humidity, updated) VALUES (?,?,?,?,?,NOW())")) { 12 | $stmt->bind_param("siddi", $data['timestamp'], $data['heating'], $data['target_temp'], $data['current_temp'], $data['humidity']); 13 | $stmt->execute(); 14 | $stmt->close(); 15 | } 16 | } 17 | $db->close(); 18 | } catch (Exception $e) { 19 | $errors[] = ("DB connection error! " . $e->getMessage() . "."); 20 | } 21 | 22 | ?> -------------------------------------------------------------------------------- /nest-api-php-workaround-login.php: -------------------------------------------------------------------------------- 1 | $o->urls->transport_url, 19 | 'access_token' => $o->access_token, 20 | 'user' => $o->user, 21 | 'userid' => $o->userid, 22 | 'cache_expiration' => strtotime($o->expires_in) 23 | ); 24 | 25 | file_put_contents($cache_file, serialize($vars)); 26 | echo "Done.\n"; 27 | echo "Access token will expire on $o->expires_in. You will need to re-execute this script before then.\n"; 28 | -------------------------------------------------------------------------------- /nestgraph-screenshot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chriseng/nestgraph/080426a0a8b25d7e0862493509688329144489b5/nestgraph-screenshot.png -------------------------------------------------------------------------------- /test.php: -------------------------------------------------------------------------------- 1 | getStatus(); 13 | print_r($status); 14 | 15 | $infos = $nest->getDeviceInfo(); 16 | print_r($infos); 17 | 18 | stuff_we_care_about($infos); 19 | 20 | function stuff_we_care_about($info) { 21 | echo "Heating : "; 22 | printf("%s\n", ($info->current_state->heat == 1 ? 1 : 0)); 23 | echo "Timestamp : "; 24 | printf("%s\n", $info->network->last_connection); 25 | echo "Target temperature : "; 26 | printf("%.02f\n", $info->target->temperature); 27 | echo "Current temperature : "; 28 | printf("%.02f\n", $info->current_state->temperature); 29 | echo "Current humidity : "; 30 | printf("%d\n", $info->current_state->humidity); 31 | 32 | } 33 | 34 | function c_to_f($c) { 35 | return ($c * 1.8) + 32; 36 | } 37 | 38 | --------------------------------------------------------------------------------