├── .gitignore ├── LICENSE.md ├── MANIFEST.in ├── README.rst ├── housecanary ├── __init__.py ├── apiclient.py ├── authentication.py ├── constants.py ├── excel │ ├── __init__.py │ ├── analytics_data_excel.py │ └── utilities.py ├── exceptions.py ├── hc_api_excel_concat │ ├── README.rst │ ├── __init__.py │ └── hc_api_excel_concat.py ├── hc_api_export │ ├── README.rst │ ├── __init__.py │ └── hc_api_export.py ├── object.py ├── output.py ├── requestclient.py ├── response.py └── utilities.py ├── notebooks └── using-test-credentials.ipynb ├── requirements.txt ├── sample_input ├── sample-input-blocks.csv ├── sample-input-city-state.csv ├── sample-input-msas.csv ├── sample-input-slugs.csv ├── sample-input-zipcodes.csv └── sample-input.csv ├── setup.py ├── tests ├── __init__.py ├── test_apiclient.py ├── test_excel_utilities.py ├── test_files │ ├── test_excel.xlsx │ └── test_input.csv ├── test_hc_api_excel_concat.py ├── test_hc_api_export.py ├── test_object.py ├── test_response.py └── test_utilities.py └── tox.ini /.gitignore: -------------------------------------------------------------------------------- 1 | # Compiled python modules. 2 | *.pyc 3 | 4 | # Setuptools distribution folder. 5 | /dist/ 6 | /build/ 7 | /.eggs/ 8 | 9 | # Python egg metadata, regenerated from source files by setuptools. 10 | /*.egg-info 11 | 12 | /notebooks/.ipynb_checkpoints 13 | 14 | .coverage 15 | /cover/ -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2016 HouseCanary, Inc 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include README.rst -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | HouseCanary API Python Client 2 | ============================= 3 | 4 | .. image:: https://img.shields.io/pypi/pyversions/housecanary.svg 5 | :target: https://pypi.python.org/pypi/housecanary 6 | 7 | The `HouseCanary `_ API Python Client provides an easy interface to call the HouseCanary API. 8 | 9 | 10 | API documentation 11 | ----------------- 12 | 13 | Full documentation is available at https://api-docs.housecanary.com 14 | 15 | Installation 16 | ------------ 17 | 18 | To install: 19 | 20 | :: 21 | 22 | pip install housecanary 23 | 24 | Basic Usage 25 | ----------- 26 | 27 | .. code:: python 28 | 29 | import housecanary 30 | client = housecanary.ApiClient("my_api_key", "my_api_secret") 31 | result = client.property.value(("10216 N Willow Ave", "64157")) 32 | 33 | # result is an instance of housecanary.response.Response 34 | print result.json() 35 | 36 | Authentication 37 | -------------- 38 | 39 | When you create an instance of an ApiClient, you need to give it your 40 | API key and secret. You can manage these in your settings page at 41 | https://valuereport.housecanary.com/#/settings/api-settings. 42 | 43 | You can pass these values to the ApiClient constructor: 44 | 45 | .. code:: python 46 | 47 | client = housecanary.ApiClient("my_api_key", "my_api_secret") 48 | 49 | Alternatively, instead of passing in your key and secret to the 50 | constructor, you can store them in the following environment variables: 51 | 52 | - HC\_API\_KEY 53 | - HC\_API\_SECRET 54 | 55 | Creating an instance of ApiClient with no arguments will read your key 56 | and secret from those environment variables: 57 | 58 | .. code:: python 59 | 60 | client = housecanary.ApiClient() 61 | 62 | Usage Details 63 | ------------- 64 | 65 | Endpoint methods 66 | ~~~~~~~~~~~~~~~~ 67 | 68 | The ApiClient class provides a few wrappers which contain 69 | various methods for calling the different API endpoints. 70 | 71 | The property wrapper is used for calling the Analytics API Property endpoints as well 72 | as the Value Report and Rental Report endpoints. 73 | 74 | The block wrapper is used for calling the Analytics API Block endpoints. 75 | 76 | The zip wrapper is used for calling the Analytics API Zip endpoints. 77 | 78 | The msa wrapper is used for calling the Analytics API MSA endpoints. 79 | 80 | 81 | Property Endpoints 82 | ~~~~~~~~~~~~~~~~~~ 83 | 84 | Analytics API Property Endpoints: 85 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 86 | 87 | - **block_histogram_baths** 88 | - **block_histogram_beds** 89 | - **block_histogram_building_area** 90 | - **block_histogram_value** 91 | - **block_histogram_value_sqft** 92 | - **block_rental_value_distribution** 93 | - **block_value_distribution** 94 | - **block_value_ts** 95 | - **block_value_ts_historical** 96 | - **block_value_ts_forecast** 97 | - **census** 98 | - **details** 99 | - **flood** 100 | - **ltv** 101 | - **ltv_details** 102 | - **mortgage_lien** 103 | - **msa_details** 104 | - **msa_hpi_ts** 105 | - **msa_hpi_ts_forecast** 106 | - **msa_hpi_ts_historical** 107 | - **nod** 108 | - **owner_occupied** 109 | - **rental_value** 110 | - **rental_value_within_block** 111 | - **sales_history** 112 | - **school** 113 | - **value** 114 | - **value_forecast** 115 | - **value_within_block** 116 | - **zip_details** 117 | - **zip_hpi_forecast** 118 | - **zip_hpi_historical** 119 | - **zip_hpi_ts** 120 | - **zip_hpi_ts_forecast** 121 | - **zip_hpi_ts_historical** 122 | - **zip_volatility** 123 | - **component_mget** 124 | 125 | Value Report API Endpoint: 126 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ 127 | 128 | - **value_report** 129 | 130 | Rental Report API Endpoint: 131 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ 132 | 133 | - **rental_report** 134 | 135 | 136 | Args: 137 | ^^^^^ 138 | 139 | All of the Analytics API property endpoint methods take an 140 | ``data`` argument. ``data`` can be in the following forms: 141 | 142 | A dict like: 143 | 144 | .. code:: python 145 | 146 | {"address": "82 County Line Rd", "zipcode": "72173", "meta": "someID"} 147 | 148 | Or 149 | 150 | .. code:: python 151 | 152 | {"address": "82 County Line Rd", "city": "San Francisco", "state": "CA", "meta": "someID"} 153 | 154 | Or 155 | 156 | .. code:: python 157 | 158 | {"slug": "123-Example-St-San-Francisco-CA-94105"} 159 | 160 | A list of dicts as specified above: 161 | 162 | .. code:: python 163 | 164 | [{"address": "82 County Line Rd", "zipcode": "72173", "meta": "someID"}, 165 | {"address": "43 Valmonte Plaza", "zipcode": "90274", "meta": "someID2"}] 166 | 167 | A single string representing a slug: 168 | 169 | .. code:: python 170 | 171 | "123-Example-St-San-Francisco-CA-94105" 172 | 173 | A tuple in the form of (address, zipcode, meta) like: 174 | 175 | .. code:: python 176 | 177 | ("82 County Line Rd", "72173", "someID") 178 | 179 | A list of (address, zipcode, meta) tuples like: 180 | 181 | .. code:: python 182 | 183 | [("82 County Line Rd", "72173", "someID"), 184 | ("43 Valmonte Plaza", "90274", "someID2")] 185 | 186 | Using a tuple only supports address, zipcode and meta. To specify city, state, unit or slug, 187 | please use a dict. 188 | 189 | The "meta" field is always optional. 190 | 191 | The available keys in the dict are: 192 | - address (required if no slug) 193 | - slug (required if no address) 194 | - zipcode (optional) 195 | - unit (optional) 196 | - city (optional) 197 | - state (optional) 198 | - meta (optional) 199 | - client_value (optional, for ``value_within_block`` and ``rental_value_within_block``) 200 | - client_value_sqft (optional, for ``value_within_block`` and ``rental_value_within_block``) 201 | 202 | All of the property endpoint methods return a PropertyResponse object 203 | (or ValueReportResponse or RentalReportResponse) or 204 | the output of a custom OutputGenerator if one was specified in the constructor. 205 | 206 | **Examples:** 207 | 208 | 209 | .. code:: python 210 | 211 | client = housecanary.ApiClient() 212 | result = client.property.value([("10216 N Willow Ave", "64157"), ("82 County Line Rd", "72173")]) 213 | 214 | result = client.property.value({"address": "10216 N Willow Ave", "city": "San Francisco", "state": "CA"}) 215 | 216 | result = client.property.value("123-Example-St-San-Francisco-CA-94105") 217 | 218 | 219 | Component_mget endpoint 220 | ^^^^^^^^^^^^^^^^^^^^^^^ 221 | 222 | You may want to retrieve data from multiple Analytics API endpoints in one request. 223 | In this case, you can use the ``component_mget`` method. 224 | The ``component_mget`` method takes an ``address_data`` argument just like the other endpoint methods. 225 | Pass in a list of Analytics API property endpoint names as the second argument. 226 | Note that ``value_report`` and ``rental_report`` cannot be included. 227 | 228 | **Example:** 229 | 230 | 231 | .. code:: python 232 | 233 | client = housecanary.ApiClient() 234 | result = client.property.component_mget(("10216 N Willow Ave", "64157"), ["property/school", "property/census", "property/details"]) 235 | 236 | 237 | Value Report: 238 | ^^^^^^^^^^^^^ 239 | 240 | The ``value_report`` method behaves differently than the other endpoint 241 | methods. It only supports one address at a time, and it takes some 242 | extra, optional parameters: 243 | 244 | Args: 245 | - *address* (str) 246 | - *zipcode* (str) 247 | 248 | Kwargs: 249 | - *report\_type* - "full" or "summary". Optional. Default is "full" 250 | - *format\_type* - "json", "pdf", "xlsx" or "all". Optional. Default is "json" 251 | 252 | **Example:** 253 | 254 | 255 | .. code:: python 256 | 257 | client = housecanary.ApiClient() 258 | # get Value Report in JSON format with "summary" report_type. 259 | result = client.property.value_report("10216 N Willow Ave", "64157", "summary", "json") 260 | # print the JSON output 261 | print result.json() 262 | 263 | # get Value Report in PDF format with "full" report_type. 264 | result = client.property.value_report("10216 N Willow Ave", "64157", format_type="pdf") 265 | # result is binary data of the PDF. 266 | 267 | Rental Report: 268 | ^^^^^^^^^^^^^^ 269 | 270 | The ``rental_report`` method is for calling the Rental Report API. It only supports one address at a time. 271 | 272 | Args: 273 | - *address* (str) 274 | - *zipcode* (str) 275 | 276 | Kwargs: 277 | - *format\_type* - "json", "xlsx" or "all". Optional. Default is "json" 278 | 279 | Learn more about the various endpoints in the `API docs. `_ 280 | 281 | 282 | Block Endpoints 283 | ~~~~~~~~~~~~~~~ 284 | 285 | Analytics API Block Endpoints: 286 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 287 | 288 | - **histogram_baths** 289 | - **histogram_beds** 290 | - **histogram_building_area** 291 | - **histogram_value** 292 | - **histogram_value_sqft** 293 | - **rental_value_distribution** 294 | - **value_distribution** 295 | - **value_ts** 296 | - **value_ts_forecast** 297 | - **value_ts_historical** 298 | - **component_mget** 299 | 300 | Args: 301 | ^^^^^ 302 | 303 | All of the Analytics API block endpoints take a ``block_data`` argument. 304 | ``block_data`` can be in the following forms: 305 | 306 | A dict with a ``block_id`` like: 307 | 308 | .. code:: python 309 | 310 | {"block_id": "060750615003005", "meta": "someId"} 311 | 312 | For histogram endpoints you can include the ``num_bins`` key: 313 | 314 | .. code:: python 315 | 316 | {"block_id": "060750615003005", "num_bins": 5, "meta": "someId"} 317 | 318 | For time series and distribution endpoints you can include the ``property_type`` key: 319 | 320 | .. code:: python 321 | 322 | {"block_id": "060750615003005", "property_type": "SFD", "meta": "someId"} 323 | 324 | A list of dicts as specified above: 325 | 326 | .. code:: python 327 | 328 | [{"block_id": "012345678901234", "meta": "someId"}, {"block_id": "012345678901234", "meta": "someId2}] 329 | 330 | A single string representing a ``block_id``: 331 | 332 | .. code:: python 333 | 334 | "012345678901234" 335 | 336 | A list of ``block_id`` strings: 337 | 338 | .. code:: python 339 | 340 | ["012345678901234", "060750615003005"] 341 | 342 | The "meta" field is always optional. 343 | 344 | See https://api-docs.housecanary.com/#analytics-api-block-level for more details 345 | on the available parameters such as ``num_bins`` and ``property_type``. 346 | 347 | All of the block endpoint methods return a BlockResponse, 348 | or the output of a custom OutputGenerator if one was specified in the constructor. 349 | 350 | 351 | **Examples:** 352 | 353 | .. code:: python 354 | 355 | client = housecanary.ApiClient() 356 | result = client.block.histogram_baths("060750615003005") 357 | 358 | result = client.block.histogram_baths({"block_id": "060750615003005", "num_bins": 5}) 359 | 360 | result = client.block.value_ts({"block_id": "060750615003005", "property_type": "SFD"}) 361 | 362 | result = client.block.value_ts([{"block_id": "060750615003005", "property_type": "SFD"}, {"block_id": "012345678901234", "property_type": "SFD"}]) 363 | 364 | result = client.block.value_distribution(["012345678901234", "060750615003005"]) 365 | 366 | 367 | Zip Endpoints 368 | ~~~~~~~~~~~~~~~ 369 | 370 | Analytics API Zip Endpoints: 371 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 372 | 373 | - **details** 374 | - **hpi_forecast** 375 | - **hpi_historical** 376 | - **hpi_ts** 377 | - **hpi_ts_forecast** 378 | - **hpi_ts_historical** 379 | - **volatility** 380 | - **component_mget** 381 | 382 | Args: 383 | ^^^^^ 384 | 385 | All of the Analytics API zip endpoints take a ``zip_data`` argument. 386 | ``zip_data`` can be in the following forms: 387 | 388 | A dict with a ``zipcode`` like: 389 | 390 | .. code:: python 391 | 392 | {"zipcode": "90274", "meta": "someId"} 393 | 394 | A list of dicts as specified above: 395 | 396 | .. code:: python 397 | 398 | [{"zipcode": "90274", "meta": "someId"}, {"zipcode": "01960", "meta": "someId2}] 399 | 400 | A single string representing a ``zipcode``: 401 | 402 | .. code:: python 403 | 404 | "90274" 405 | 406 | A list of ``zipcode`` strings: 407 | 408 | .. code:: python 409 | 410 | ["90274", "01960"] 411 | 412 | The "meta" field is always optional. 413 | 414 | All of the zip endpoint methods return a ZipCodeResponse, 415 | or the output of a custom OutputGenerator if one was specified in the constructor. 416 | 417 | 418 | **Examples:** 419 | 420 | .. code:: python 421 | 422 | client = housecanary.ApiClient() 423 | result = client.zip.details("90274") 424 | 425 | result = client.zip.details({"zipcode": "90274", "meta": "someId"}) 426 | 427 | result = client.zip.details([{"zipcode": "90274", "meta": "someId"}, {"zipcode": "01960", "meta": "someId2"}]) 428 | 429 | result = client.zip.details(["90274", "01960"]) 430 | 431 | 432 | MSA Endpoints 433 | ~~~~~~~~~~~~~~~ 434 | 435 | Analytics API MSA Endpoints: 436 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 437 | 438 | - **details** 439 | - **hpi_ts** 440 | - **hpi_ts_forecast** 441 | - **hpi_ts_historical** 442 | - **component_mget** 443 | 444 | Args: 445 | ^^^^^ 446 | 447 | All of the Analytics API MSA endpoints take an ``msa_data`` argument. 448 | ``msa_data`` can be in the following forms: 449 | 450 | A dict with an ``msa`` like: 451 | 452 | .. code:: python 453 | 454 | {"msa": "41860", "meta": "someId"} 455 | 456 | A list of dicts as specified above: 457 | 458 | .. code:: python 459 | 460 | [{"msa": "41860", "meta": "someId"}, {"msa": "40928", "meta": "someId2}] 461 | 462 | A single string representing a ``msa``: 463 | 464 | .. code:: python 465 | 466 | "41860" 467 | 468 | A list of ``msa`` strings: 469 | 470 | .. code:: python 471 | 472 | ["41860", "40928"] 473 | 474 | The "meta" field is always optional. 475 | 476 | All of the msa endpoint methods return an MsaResponse, 477 | or the output of a custom OutputGenerator if one was specified in the constructor. 478 | 479 | 480 | **Examples:** 481 | 482 | .. code:: python 483 | 484 | client = housecanary.ApiClient() 485 | result = client.msa.details("41860") 486 | 487 | result = client.msa.details({"msa": "90274", "meta": "someId"}) 488 | 489 | result = client.msa.details([{"msa": "41860", "meta": "someId"}, {"msa": "40928", "meta": "someId2"}]) 490 | 491 | result = client.msa.details(["41860", "40928"]) 492 | 493 | 494 | Response Objects 495 | ~~~~~~~~~~~~~~~~ 496 | 497 | Response 498 | ^^^^^^^^ 499 | 500 | Response is a base class for encapsulating an HTTP response from the 501 | HouseCanary API. 502 | 503 | **Properties:** 504 | 505 | 506 | - **endpoint\_name** - Gets the endpoint name of the original request 507 | - **response** - Gets the underlying response object. 508 | 509 | **Methods:** 510 | 511 | 512 | - **json()** - Gets the body of the response from the API as json. 513 | - **has\_object\_error()** - Returns true if any requested objects had 514 | a business logic error, otherwise returns false. 515 | - **get\_object\_errors()** - Gets a list of business error message 516 | strings for each of the requested objects that had a business error. 517 | If there was no error, returns an empty list. 518 | - **objects()** - Overridden in subclasses. 519 | - **rate_limits** - Returns a list of rate limit information 520 | 521 | PropertyResponse 522 | ^^^^^^^^^^^^^^^^ 523 | 524 | A subclass of Response, this is returned for all property endpoints 525 | except for ``value_report`` and ``rental_report``. 526 | 527 | **Methods:** 528 | 529 | 530 | - **objects()** - Gets a list of Property objects for the requested 531 | properties, each containing the object's returned json data from the 532 | API. 533 | - **properties()** - An alias for the objects() method. 534 | 535 | BlockResponse 536 | ^^^^^^^^^^^^^^^^ 537 | 538 | A subclass of Response, this is returned for all block endpoints. 539 | 540 | **Methods:** 541 | 542 | 543 | - **objects()** - Gets a list of Block objects for the requested 544 | blocks, each containing the object's returned json data from the 545 | API. 546 | - **blocks()** - An alias for the objects() method. 547 | 548 | ZipCodeResponse 549 | ^^^^^^^^^^^^^^^^ 550 | 551 | A subclass of Response, this is returned for all zip endpoints. 552 | 553 | **Methods:** 554 | 555 | 556 | - **objects()** - Gets a list of ZipCode objects for the requested 557 | zipcodes, each containing the object's returned json data from the 558 | API. 559 | - **zipcodes()** - An alias for the objects() method. 560 | 561 | MsaResponse 562 | ^^^^^^^^^^^^^^^^ 563 | 564 | A subclass of Response, this is returned for all msa endpoints. 565 | 566 | **Methods:** 567 | 568 | 569 | - **objects()** - Gets a list of Msa objects for the requested 570 | msas, each containing the object's returned json data from the 571 | API. 572 | - **msas()** - An alias for the objects() method. 573 | 574 | HouseCanaryObject 575 | ^^^^^^^^^^^^^^^^^ 576 | 577 | Base class for various types of objects returned from the HouseCanary 578 | API. Currently, only the Property subclass is implemented. 579 | 580 | **Properties:** 581 | 582 | 583 | - **component\_results** - a list of ComponentResult objects that 584 | contain data and error information for each endpoint requested for 585 | this HouseCanaryObject. 586 | 587 | **Methods:** 588 | 589 | 590 | - **has\_error()** - Returns a boolean of whether there was a business 591 | logic error fetching data for any components for this object. 592 | - **get\_errors()** - If there was a business error fetching data for 593 | any components for this object, returns the error messages. 594 | 595 | Property 596 | ^^^^^^^^ 597 | 598 | A subclass of HouseCanaryObject, the Property represents a single 599 | address and it's returned data. 600 | 601 | **Properties:** 602 | 603 | 604 | - **address** 605 | - **zipcode** 606 | - **zipcode\_plus4** 607 | - **address\_full** 608 | - **city** 609 | - **country\_fips** 610 | - **lat** 611 | - **lng** 612 | - **state** 613 | - **unit** 614 | - **meta** 615 | 616 | **Example:** 617 | 618 | 619 | .. code:: python 620 | 621 | result = client.property.value(("123 Main St", "01234", "meta information")) 622 | p = result.properties()[0] 623 | print p.address 624 | # "123 Main St" 625 | print p.zipcode 626 | # "01234" 627 | print p.meta 628 | # "meta information" 629 | value_result = p.component_results[0] 630 | print value_result.component_name 631 | # 'property/value' 632 | print value_result.api_code 633 | # 0 634 | print value_result.api_code_description 635 | # 'ok' 636 | print value_result.json_data 637 | # {u'value': {u'price_upr': 1575138.0, u'price_lwr': 1326125.0, u'price_mean': 1450632.0, u'fsd': 0.086}} 638 | print p.has_error() 639 | # False 640 | print p.get_errors() 641 | # [] 642 | 643 | 644 | Block 645 | ^^^^^ 646 | 647 | A subclass of HouseCanaryObject, the Block represents a single 648 | block and it's returned data. 649 | 650 | **Properties:** 651 | 652 | 653 | - **block_id** 654 | - **property_type** 655 | - **meta** 656 | 657 | **Example:** 658 | 659 | 660 | .. code:: python 661 | 662 | result = client.block.value_ts("060750615003005") 663 | b = result.blocks()[0] 664 | print b.block_id 665 | # "060750615003005" 666 | print b.meta 667 | # "meta information" 668 | value_result = b.component_results[0] 669 | print value_result.component_name 670 | # 'block/value_ts' 671 | print value_result.api_code 672 | # 0 673 | print value_result.api_code_description 674 | # 'ok' 675 | print value_result.json_data 676 | # [...data...] 677 | print b.has_error() 678 | # False 679 | print b.get_errors() 680 | # [] 681 | 682 | 683 | ZipCode 684 | ^^^^^^^ 685 | 686 | A subclass of HouseCanaryObject, the ZipCode represents a single 687 | zipcode and it's returned data. 688 | 689 | **Properties:** 690 | 691 | 692 | - **zipcode** 693 | - **meta** 694 | 695 | **Example:** 696 | 697 | 698 | .. code:: python 699 | 700 | result = client.zip.details("90274") 701 | z = result.zipcodes()[0] 702 | print z.zipcode 703 | # "90274" 704 | print z.meta 705 | # "meta information" 706 | details_result = z.component_results[0] 707 | print details_result.component_name 708 | # 'zip/details' 709 | print details_result.api_code 710 | # 0 711 | print details_result.api_code_description 712 | # 'ok' 713 | print details_result.json_data 714 | # [...data...] 715 | print z.has_error() 716 | # False 717 | print z.get_errors() 718 | # [] 719 | 720 | 721 | Msa 722 | ^^^^^^^ 723 | 724 | A subclass of HouseCanaryObject, the Msa represents a single 725 | Metropolitan Statistical Area and it's returned data. 726 | 727 | **Properties:** 728 | 729 | 730 | - **msa** 731 | - **meta** 732 | 733 | **Example:** 734 | 735 | 736 | .. code:: python 737 | 738 | result = client.msa.details("41860") 739 | m = result.msas()[0] 740 | print m.msa 741 | # "41860" 742 | print m.meta 743 | # "meta information" 744 | details_result = m.component_results[0] 745 | print details_result.component_name 746 | # 'msa/details' 747 | print details_result.api_code 748 | # 0 749 | print details_result.api_code_description 750 | # 'ok' 751 | print details_result.json_data 752 | # [...data...] 753 | print m.has_error() 754 | # False 755 | print m.get_errors() 756 | # [] 757 | 758 | 759 | ValueReportResponse 760 | ^^^^^^^^^^^^^^^^^^^ 761 | 762 | A subclass of Response, this is the object returned for the 763 | ``value_report`` endpoint when "json" format\_type is used. It simply 764 | returns the JSON data of the Value Report. 765 | 766 | **Example:** 767 | 768 | 769 | .. code:: python 770 | 771 | result = client.property.value_report("123 Main St", "01234") 772 | print result.json() 773 | 774 | RentalReportResponse 775 | ^^^^^^^^^^^^^^^^^^^^ 776 | 777 | A subclass of Response, this is the object returned for the 778 | ``rental_report`` endpoint when "json" format\_type is used. It simply 779 | returns the JSON data of the Rental Report. 780 | 781 | **Example:** 782 | 783 | 784 | .. code:: python 785 | 786 | result = client.property.rental_report("123 Main St", "01234") 787 | print result.json() 788 | 789 | Command Line Tools 790 | --------------------------- 791 | When you install this package, a couple command line tools are included and installed on your PATH. 792 | 793 | - `HouseCanary Analytics API Export `_ 794 | - `HouseCanary API Excel Concat `_ 795 | 796 | Running Tests 797 | --------------------------- 798 | 799 | To run the unit test suite: 800 | 801 | :: 802 | 803 | python setup.py nosetests --with-coverage --cover-package=housecanary 804 | 805 | During unit tests, all API requests are mocked. 806 | You can run the unit tests with real API requests by doing the following: 807 | 808 | 809 | - Update `URL_PREFIX` in `constants.py` to point to a test or dev environment 810 | - Obtain an account for that environment that has permissions to all API components 811 | - Obtain an API Key and Secret and put them in `HC_API_KEY` and `HC_API_SECRET` environment variables 812 | - Run the test suite as: 813 | 814 | :: 815 | 816 | HC_API_CALLS=true python setup.py nosetests 817 | 818 | **Note:** This setting will be ignored and API requests will be mocked if `URL_PREFIX` is pointing to Production. 819 | 820 | License 821 | ------- 822 | 823 | This API Client Library is made available under the MIT License: 824 | 825 | The MIT License (MIT) 826 | 827 | Copyright (c) 2017 HouseCanary, Inc 828 | 829 | Permission is hereby granted, free of charge, to any person obtaining a 830 | copy of this software and associated documentation files (the 831 | "Software"), to deal in the Software without restriction, including 832 | without limitation the rights to use, copy, modify, merge, publish, 833 | distribute, sublicense, and/or sell copies of the Software, and to 834 | permit persons to whom the Software is furnished to do so, subject to 835 | the following conditions: 836 | 837 | The above copyright notice and this permission notice shall be included 838 | in all copies or substantial portions of the Software. 839 | 840 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS 841 | OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 842 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 843 | IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY 844 | CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, 845 | TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 846 | SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 847 | 848 | For the avoidance of doubt, the above license does not apply to 849 | HouseCanary's proprietary software code or APIs, or to any data, 850 | analytics or reports made available by HouseCanary from time to time, 851 | all of which may be licensed pursuant to a separate written agreement 852 | -------------------------------------------------------------------------------- /housecanary/__init__.py: -------------------------------------------------------------------------------- 1 | __version__ = '0.7.1' 2 | 3 | from housecanary.apiclient import ApiClient 4 | from housecanary.excel import export_analytics_data_to_excel 5 | from housecanary.excel import export_analytics_data_to_csv 6 | from housecanary.excel import concat_excel_reports 7 | from housecanary.excel import utilities as excel_utilities 8 | -------------------------------------------------------------------------------- /housecanary/apiclient.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | import os 3 | from builtins import object 4 | from housecanary.output import ResponseOutputGenerator 5 | from housecanary.requestclient import RequestClient 6 | import housecanary.exceptions 7 | import housecanary.constants as constants 8 | from requests.auth import HTTPBasicAuth 9 | 10 | 11 | class ApiClient(object): 12 | """Base class for making API calls""" 13 | 14 | def __init__(self, auth_key=None, auth_secret=None, version=None, request_client=None, 15 | output_generator=None, auth=None): 16 | """ 17 | auth_key and auth_secret can be passed in as parameters or 18 | pulled automatically from the following environment variables: 19 | HC_API_KEY -- Your HouseCanary API auth key 20 | HC_API_SECRET -- Your HouseCanary API secret 21 | 22 | Passing in the key and secret as parameters will take precedence over environment variables. 23 | 24 | Args: 25 | auth_key - Optional. The HouseCanary API auth key. 26 | auth_secret - Optional. The HouseCanary API secret. 27 | version (str) - Optional. The API version to use, in the form "v2", for example. 28 | Default is "v2". 29 | request_client - Optional. An instance of a class that is responsible for 30 | making http requests. It must implement a `post` method. 31 | Default is RequestClient. 32 | output_generator - Optional. An instance of an OutputGenerator that implements 33 | a `process_response` method. Can be used to implement custom 34 | serialization of the response data from the request client. 35 | Default is ResponseOutputGenerator. 36 | authenticator - Optional. An instance of a requests.auth.AuthBase implementation 37 | for providing authentication to the request. 38 | Default is requests.auth.HTTPBasicAuth. 39 | """ 40 | 41 | self._auth_key = auth_key or os.getenv('HC_API_KEY') 42 | self._auth_secret = auth_secret or os.getenv('HC_API_SECRET') 43 | 44 | self._version = version or constants.DEFAULT_VERSION 45 | 46 | # user can pass in a custom request_client 47 | self._request_client = request_client 48 | 49 | # if no request_client provided, use the defaults. 50 | if self._request_client is None: 51 | # allow using custom OutputGenerator or Authenticator with the RequestClient 52 | _output_generator = output_generator or ResponseOutputGenerator() 53 | _auth = auth or HTTPBasicAuth(self._auth_key, self._auth_secret) 54 | self._request_client = RequestClient(_output_generator, _auth) 55 | 56 | self.property = PropertyComponentWrapper(self) 57 | self.block = BlockComponentWrapper(self) 58 | self.zip = ZipComponentWrapper(self) 59 | self.msa = MsaComponentWrapper(self) 60 | 61 | def fetch(self, endpoint_name, identifier_input, query_params=None): 62 | """Calls this instance's request_client's post method with the 63 | specified component endpoint 64 | 65 | Args: 66 | - endpoint_name (str) - The endpoint to call like "property/value". 67 | - identifier_input - One or more identifiers to request data for. An identifier can 68 | be in one of these forms: 69 | 70 | - A list of property identifier dicts: 71 | - A property identifier dict can contain the following keys: 72 | (address, zipcode, unit, city, state, slug, meta). 73 | One of 'address' or 'slug' is required. 74 | 75 | Ex: [{"address": "82 County Line Rd", 76 | "zipcode": "72173", 77 | "meta": "some ID"}] 78 | 79 | A slug is a URL-safe string that identifies a property. 80 | These are obtained from HouseCanary. 81 | 82 | Ex: [{"slug": "123-Example-St-San-Francisco-CA-94105"}] 83 | 84 | - A list of dicts representing a block: 85 | - A block identifier dict can contain the following keys: 86 | (block_id, num_bins, property_type, meta). 87 | 'block_id' is required. 88 | 89 | Ex: [{"block_id": "060750615003005", "meta": "some ID"}] 90 | 91 | - A list of dicts representing a zipcode: 92 | 93 | Ex: [{"zipcode": "90274", "meta": "some ID"}] 94 | 95 | - A list of dicts representing an MSA: 96 | 97 | Ex: [{"msa": "41860", "meta": "some ID"}] 98 | 99 | The "meta" field is always optional. 100 | 101 | Returns: 102 | A Response object, or the output of a custom OutputGenerator 103 | if one was specified in the constructor. 104 | """ 105 | 106 | endpoint_url = constants.URL_PREFIX + "/" + self._version + "/" + endpoint_name 107 | 108 | if query_params is None: 109 | query_params = {} 110 | 111 | if len(identifier_input) == 1: 112 | # If only one identifier specified, use a GET request 113 | query_params.update(identifier_input[0]) 114 | return self._request_client.get(endpoint_url, query_params) 115 | 116 | # when more than one address, use a POST request 117 | return self._request_client.post(endpoint_url, identifier_input, query_params) 118 | 119 | def fetch_synchronous(self, endpoint_name, query_params=None): 120 | """Calls this instance's request_client's get method with the 121 | specified component endpoint""" 122 | 123 | endpoint_url = constants.URL_PREFIX + "/" + self._version + "/" + endpoint_name 124 | 125 | if query_params is None: 126 | query_params = {} 127 | 128 | return self._request_client.get(endpoint_url, query_params) 129 | 130 | 131 | class ComponentWrapper(object): 132 | def __init__(self, api_client=None): 133 | """ 134 | Args: 135 | - api_client - An instance of ApiClient 136 | """ 137 | self._api_client = api_client 138 | 139 | def _convert_to_identifier_json(self, identifier_data): 140 | """Override in subclasses""" 141 | raise NotImplementedError() 142 | 143 | def get_identifier_input(self, identifier_data): 144 | """Convert the various formats of input identifier_data into 145 | the proper json format expected by the ApiClient fetch method, 146 | which is a list of dicts.""" 147 | 148 | identifier_input = [] 149 | 150 | if isinstance(identifier_data, list) and len(identifier_data) > 0: 151 | # if list, convert each item in the list to json 152 | for address in identifier_data: 153 | identifier_input.append(self._convert_to_identifier_json(address)) 154 | else: 155 | identifier_input.append(self._convert_to_identifier_json(identifier_data)) 156 | 157 | return identifier_input 158 | 159 | def fetch_identifier_component(self, endpoint_name, identifier_data, query_params=None): 160 | """Common method for handling parameters before passing to api_client""" 161 | 162 | if query_params is None: 163 | query_params = {} 164 | 165 | identifier_input = self.get_identifier_input(identifier_data) 166 | 167 | return self._api_client.fetch(endpoint_name, identifier_input, query_params) 168 | 169 | 170 | class PropertyComponentWrapper(ComponentWrapper): 171 | """Property specific components 172 | 173 | All the component methods of this class (except value_report and rental_report) 174 | take data as a parameter. data can be in the following forms: 175 | 176 | - A dict like: 177 | {"address": "82 County Line Rd", "zipcode": "72173", "meta": "someID"} 178 | or 179 | {"address": "82 County Line Rd", "city": "San Francisco", "state": "CA", "meta": "someID"} 180 | or 181 | {"slug": "123-Example-St-San-Francisco-CA-94105"} 182 | 183 | - A list of dicts as specified above: 184 | [{"address": "82 County Line Rd", "zipcode": "72173", "meta": "someID"}, 185 | {"address": "43 Valmonte Plaza", "zipcode": "90274", "meta": "someID2"}] 186 | 187 | - A single string representing a slug: 188 | "123-Example-St-San-Francisco-CA-94105" 189 | 190 | - A tuple in the form of (address, zipcode, meta) like: 191 | ("82 County Line Rd", "72173", "someID") 192 | 193 | - A list of (address, zipcode, meta) tuples like: 194 | [("82 County Line Rd", "72173", "someID"), 195 | ("43 Valmonte Plaza", "90274", "someID2")] 196 | 197 | Using a tuple only supports address, zipcode and meta. To specify city, state, unit or slug, 198 | please use a dict. 199 | 200 | The "meta" field is always optional. 201 | 202 | The available keys in the dict are: 203 | - address (required if no slug) 204 | - slug (required if no address) 205 | - zipcode (optional) 206 | - unit (optional) 207 | - city (optional) 208 | - state (optional) 209 | - meta (optional) 210 | - client_value (optional, for "value_within_block" and "rental_value_within_block") 211 | - client_value_sqft (optional, for "value_within_block" and "rental_value_within_block") 212 | 213 | All the component methods of this class return a PropertyResponse object, 214 | (or ValueReportResponse or RentalReportResponse) 215 | or the output of a custom OutputGenerator if one was specified in the constructor. 216 | """ 217 | 218 | def _convert_to_identifier_json(self, address_data): 219 | """Convert input address data into json format""" 220 | 221 | if isinstance(address_data, str): 222 | # allow just passing a slug string. 223 | return {"slug": address_data} 224 | 225 | if isinstance(address_data, tuple) and len(address_data) > 0: 226 | address_json = {"address": address_data[0]} 227 | if len(address_data) > 1: 228 | address_json["zipcode"] = address_data[1] 229 | if len(address_data) > 2: 230 | address_json["meta"] = address_data[2] 231 | return address_json 232 | 233 | if isinstance(address_data, dict): 234 | allowed_keys = ["address", "zipcode", "unit", "city", "state", "slug", "meta", 235 | "client_value", "client_value_sqft"] 236 | 237 | # ensure the dict does not contain any unallowed keys 238 | for key in address_data: 239 | if key not in allowed_keys: 240 | msg = "Key in address input not allowed: " + key 241 | raise housecanary.exceptions.InvalidInputException(msg) 242 | 243 | # ensure it contains an "address" key 244 | if "address" in address_data or "slug" in address_data: 245 | return address_data 246 | 247 | # if we made it here, the input was not valid. 248 | msg = ("Input is invalid. Must be a list of (address, zipcode) tuples, or a dict or list" 249 | " of dicts with each item containing at least an 'address' or 'slug' key.") 250 | raise housecanary.exceptions.InvalidInputException((msg)) 251 | 252 | def block_histogram_baths(self, data): 253 | """Call the block_histogram_baths endpoint""" 254 | return self.fetch_identifier_component("property/block_histogram_baths", data) 255 | 256 | def block_histogram_beds(self, data): 257 | """Call the block_histogram_beds endpoint""" 258 | return self.fetch_identifier_component("property/block_histogram_beds", data) 259 | 260 | def block_histogram_building_area(self, data): 261 | """Call the block_histogram_building_area endpoint""" 262 | return self.fetch_identifier_component("property/block_histogram_building_area", data) 263 | 264 | def block_histogram_value(self, data): 265 | """Call the block_histogram_value endpoint""" 266 | return self.fetch_identifier_component("property/block_histogram_value", data) 267 | 268 | def block_histogram_value_sqft(self, data): 269 | """Call the block_histogram_value_sqft endpoint""" 270 | return self.fetch_identifier_component("property/block_histogram_value_sqft", data) 271 | 272 | def block_rental_value_distribution(self, data): 273 | """Call the block_rental_value_distribution endpoint""" 274 | return self.fetch_identifier_component("property/block_rental_value_distribution", data) 275 | 276 | def block_value_distribution(self, data): 277 | """Call the block_value_distribution endpoint""" 278 | return self.fetch_identifier_component("property/block_value_distribution", data) 279 | 280 | def block_value_ts(self, data): 281 | """Call the block_value_ts endpoint""" 282 | return self.fetch_identifier_component("property/block_value_ts", data) 283 | 284 | def block_value_ts_historical(self, data): 285 | """Call the block_value_ts_historical endpoint""" 286 | return self.fetch_identifier_component("property/block_value_ts_historical", data) 287 | 288 | def block_value_ts_forecast(self, data): 289 | """Call the block_value_ts_forecast endpoint""" 290 | return self.fetch_identifier_component("property/block_value_ts_forecast", data) 291 | 292 | def census(self, data): 293 | """Call the census endpoint""" 294 | return self.fetch_identifier_component("property/census", data) 295 | 296 | def details(self, data): 297 | """Call the details endpoint""" 298 | return self.fetch_identifier_component("property/details", data) 299 | 300 | def flood(self, data): 301 | """Call the flood endpoint""" 302 | return self.fetch_identifier_component("property/flood", data) 303 | 304 | def ltv(self, data): 305 | """Call the ltv endpoint""" 306 | return self.fetch_identifier_component("property/ltv", data) 307 | 308 | def ltv_details(self, data): 309 | """Call the ltv_details endpoint""" 310 | return self.fetch_identifier_component("property/ltv_details", data) 311 | 312 | def mortgage_lien(self, data): 313 | """Call the mortgage_lien endpoint""" 314 | return self.fetch_identifier_component("property/mortgage_lien", data) 315 | 316 | def msa_details(self, data): 317 | """Call the msa_details endpoint""" 318 | return self.fetch_identifier_component("property/msa_details", data) 319 | 320 | def msa_hpi_ts(self, data): 321 | """Call the msa_hpi_ts endpoint""" 322 | return self.fetch_identifier_component("property/msa_hpi_ts", data) 323 | 324 | def msa_hpi_ts_forecast(self, data): 325 | """Call the msa_hpi_ts_forecast endpoint""" 326 | return self.fetch_identifier_component("property/msa_hpi_ts_forecast", data) 327 | 328 | def msa_hpi_ts_historical(self, data): 329 | """Call the msa_hpi_ts_historical endpoint""" 330 | return self.fetch_identifier_component("property/msa_hpi_ts_historical", data) 331 | 332 | def nod(self, data): 333 | """Call the nod endpoint""" 334 | return self.fetch_identifier_component("property/nod", data) 335 | 336 | def owner_occupied(self, data): 337 | """Call the owner_occupied endpoint""" 338 | return self.fetch_identifier_component("property/owner_occupied", data) 339 | 340 | def rental_value(self, data): 341 | """Call the rental_value endpoint""" 342 | return self.fetch_identifier_component("property/rental_value", data) 343 | 344 | def rental_value_within_block(self, data): 345 | """Call the rental_value_within_block endpoint""" 346 | return self.fetch_identifier_component("property/rental_value_within_block", data) 347 | 348 | def sales_history(self, data): 349 | """Call the sales_history endpoint""" 350 | return self.fetch_identifier_component("property/sales_history", data) 351 | 352 | def school(self, data): 353 | """Call the school endpoint""" 354 | return self.fetch_identifier_component("property/school", data) 355 | 356 | def value(self, data): 357 | """Call the value endpoint""" 358 | return self.fetch_identifier_component("property/value", data) 359 | 360 | def value_forecast(self, data): 361 | """Call the value_forecast endpoint""" 362 | return self.fetch_identifier_component("property/value_forecast", data) 363 | 364 | def value_within_block(self, data): 365 | """Call the value_within_block endpoint""" 366 | return self.fetch_identifier_component("property/value_within_block", data) 367 | 368 | def zip_details(self, data): 369 | """Call the zip_details endpoint""" 370 | return self.fetch_identifier_component("property/zip_details", data) 371 | 372 | def zip_hpi_forecast(self, data): 373 | """Call the zip_hpi_forecast endpoint""" 374 | return self.fetch_identifier_component("property/zip_hpi_forecast", data) 375 | 376 | def zip_hpi_historical(self, data): 377 | """Call the zip_hpi_historical endpoint""" 378 | return self.fetch_identifier_component("property/zip_hpi_historical", data) 379 | 380 | def zip_hpi_ts(self, data): 381 | """Call the zip_hpi_ts endpoint""" 382 | return self.fetch_identifier_component("property/zip_hpi_ts", data) 383 | 384 | def zip_hpi_ts_forecast(self, data): 385 | """Call the zip_hpi_ts_forecast endpoint""" 386 | return self.fetch_identifier_component("property/zip_hpi_ts_forecast", data) 387 | 388 | def zip_hpi_ts_historical(self, data): 389 | """Call the zip_hpi_ts_historical endpoint""" 390 | return self.fetch_identifier_component("property/zip_hpi_ts_historical", data) 391 | 392 | def zip_volatility(self, data): 393 | """Call the zip_volatility endpoint""" 394 | return self.fetch_identifier_component("property/zip_volatility", data) 395 | 396 | def component_mget(self, data, components): 397 | """Call the component_mget endpoint 398 | 399 | Args: 400 | - data - As described in the class docstring. 401 | - components - A list of strings for each component to include in the request. 402 | Example: ["property/details", "property/flood", "property/value"] 403 | """ 404 | if not isinstance(components, list): 405 | print("Components param must be a list") 406 | return 407 | 408 | query_params = {"components": ",".join(components)} 409 | 410 | return self.fetch_identifier_component( 411 | "property/component_mget", data, query_params) 412 | 413 | def value_report(self, address, zipcode, report_type="full", format_type="json"): 414 | """Call the value_report component 415 | 416 | Value Report only supports a single address. 417 | 418 | Args: 419 | - address 420 | - zipcode 421 | 422 | Kwargs: 423 | - report_type - "full" or "summary". Default is "full". 424 | - format_type - "json", "pdf", "xlsx" or "all". Default is "json". 425 | """ 426 | query_params = { 427 | "report_type": report_type, 428 | "format": format_type, 429 | "address": address, 430 | "zipcode": zipcode 431 | } 432 | 433 | return self._api_client.fetch_synchronous("property/value_report", query_params) 434 | 435 | def rental_report(self, address, zipcode, format_type="json"): 436 | """Call the rental_report component 437 | 438 | Rental Report only supports a single address. 439 | 440 | Args: 441 | - address 442 | - zipcode 443 | 444 | Kwargs: 445 | - format_type - "json", "xlsx" or "all". Default is "json". 446 | """ 447 | 448 | # only json is supported by rental report. 449 | query_params = { 450 | "format": format_type, 451 | "address": address, 452 | "zipcode": zipcode 453 | } 454 | 455 | return self._api_client.fetch_synchronous("property/rental_report", query_params) 456 | 457 | 458 | class BlockComponentWrapper(ComponentWrapper): 459 | """Block specific components 460 | 461 | All the component methods of this class 462 | take block_data as a parameter. block_data can be in the following forms: 463 | 464 | - A dict with a ``block_id`` like: 465 | {"block_id": "060750615003005", "meta": "someId"} 466 | 467 | - For histogram endpoints you can include the ``num_bins`` key: 468 | {"block_id": "060750615003005", "num_bins": 5, "meta": "someId"} 469 | 470 | - For time series and distribution endpoints you can include the ``property_type`` key: 471 | {"block_id": "060750615003005", "property_type": "SFD", "meta": "someId"} 472 | 473 | - A list of dicts as specified above: 474 | [{"block_id": "012345678901234", "meta": "someId"}, 475 | {"block_id": "012345678901234", "meta": "someId2}] 476 | 477 | - A single string representing a ``block_id``: 478 | "012345678901234" 479 | 480 | - A list of ``block_id`` strings: 481 | ["012345678901234", "060750615003005"] 482 | 483 | The "meta" field is always optional. 484 | 485 | All the component methods of this class return a BlockResponse object, 486 | or the output of a custom OutputGenerator if one was specified in the constructor. 487 | """ 488 | 489 | def _convert_to_identifier_json(self, block_data): 490 | if isinstance(block_data, str): 491 | # allow just passing a block_id string. 492 | return {"block_id": block_data} 493 | 494 | if isinstance(block_data, dict): 495 | allowed_keys = ["block_id", "num_bins", "property_type", "meta"] 496 | 497 | # ensure the dict does not contain any unallowed keys 498 | for key in block_data: 499 | if key not in allowed_keys: 500 | msg = "Key in block input not allowed: " + key 501 | raise housecanary.exceptions.InvalidInputException(msg) 502 | 503 | # ensure it contains a "block_id" key 504 | if "block_id" in block_data: 505 | return block_data 506 | 507 | # if we made it here, the input was not valid. 508 | msg = ("Input is invalid. Must be a dict or list of dicts" 509 | " with each item containing at least 'block_id' key.") 510 | raise housecanary.exceptions.InvalidInputException((msg)) 511 | 512 | def histogram_baths(self, block_data): 513 | """Call the histogram_baths endpoint""" 514 | return self.fetch_identifier_component("block/histogram_baths", block_data) 515 | 516 | def histogram_beds(self, block_data): 517 | """Call the histogram_beds endpoint""" 518 | return self.fetch_identifier_component("block/histogram_beds", block_data) 519 | 520 | def histogram_building_area(self, block_data): 521 | """Call the histogram_building_area endpoint""" 522 | return self.fetch_identifier_component("block/histogram_building_area", block_data) 523 | 524 | def histogram_value(self, block_data): 525 | """Call the histogram_value endpoint""" 526 | return self.fetch_identifier_component("block/histogram_value", block_data) 527 | 528 | def histogram_value_sqft(self, block_data): 529 | """Call the histogram_value_sqft endpoint""" 530 | return self.fetch_identifier_component("block/histogram_value_sqft", block_data) 531 | 532 | def rental_value_distribution(self, block_data): 533 | """Call the rental_value_distribution endpoint""" 534 | return self.fetch_identifier_component("block/rental_value_distribution", block_data) 535 | 536 | def value_distribution(self, block_data): 537 | """Call the value_distribution endpoint""" 538 | return self.fetch_identifier_component("block/value_distribution", block_data) 539 | 540 | def value_ts(self, block_data): 541 | """Call the value_ts endpoint""" 542 | return self.fetch_identifier_component("block/value_ts", block_data) 543 | 544 | def value_ts_forecast(self, block_data): 545 | """Call the value_ts_forecast endpoint""" 546 | return self.fetch_identifier_component("block/value_ts_forecast", block_data) 547 | 548 | def value_ts_historical(self, block_data): 549 | """Call the value_ts_historical endpoint""" 550 | return self.fetch_identifier_component("block/value_ts_historical", block_data) 551 | 552 | def component_mget(self, block_data, components): 553 | """Call the block component_mget endpoint 554 | 555 | Args: 556 | - block_data - As described in the class docstring. 557 | - components - A list of strings for each component to include in the request. 558 | Example: ["block/value_ts", "block/value_distribution"] 559 | """ 560 | if not isinstance(components, list): 561 | print("Components param must be a list") 562 | return 563 | 564 | query_params = {"components": ",".join(components)} 565 | 566 | return self.fetch_identifier_component( 567 | "block/component_mget", block_data, query_params) 568 | 569 | 570 | class ZipComponentWrapper(ComponentWrapper): 571 | """Zip specific components 572 | 573 | All of the Analytics API zip endpoints take a ``zip_data`` argument. 574 | zip_data can be in the following forms: 575 | 576 | - A dict with a ``zipcode`` like: 577 | {"zipcode": "90274", "meta": "someId"} 578 | 579 | - A list of dicts as specified above: 580 | [{"zipcode": "90274", "meta": "someId"}, {"zipcode": "01960", "meta": "someId2}] 581 | 582 | - A single string representing a ``zipcode``: 583 | "90274" 584 | 585 | - A list of ``zipcode`` strings: 586 | ["90274", "01960"] 587 | 588 | The "meta" field is always optional. 589 | 590 | All of the zip endpoint methods return a ZipResponse, 591 | or the output of a custom OutputGenerator if one was specified in the constructor. 592 | """ 593 | 594 | def _convert_to_identifier_json(self, zip_data): 595 | if isinstance(zip_data, str): 596 | # allow just passing a zipcode string. 597 | return {"zipcode": zip_data} 598 | 599 | if isinstance(zip_data, dict): 600 | allowed_keys = ["zipcode", "meta"] 601 | 602 | # ensure the dict does not contain any unallowed keys 603 | for key in zip_data: 604 | if key not in allowed_keys: 605 | msg = "Key in zip input not allowed: " + key 606 | raise housecanary.exceptions.InvalidInputException(msg) 607 | 608 | # ensure it contains a "zipcode" key 609 | if "zipcode" in zip_data: 610 | return zip_data 611 | 612 | # if we made it here, the input was not valid. 613 | msg = ("Input is invalid. Must be a dict or list of dicts" 614 | " with each item containing at least 'zipcode' key.") 615 | raise housecanary.exceptions.InvalidInputException((msg)) 616 | 617 | def details(self, zip_data): 618 | """Call the details endpoint""" 619 | return self.fetch_identifier_component("zip/details", zip_data) 620 | 621 | def hpi_forecast(self, zip_data): 622 | """Call the hpi_forecast endpoint""" 623 | return self.fetch_identifier_component("zip/hpi_forecast", zip_data) 624 | 625 | def hpi_historical(self, zip_data): 626 | """Call the hpi_historical endpoint""" 627 | return self.fetch_identifier_component("zip/hpi_historical", zip_data) 628 | 629 | def hpi_ts(self, zip_data): 630 | """Call the hpi_ts endpoint""" 631 | return self.fetch_identifier_component("zip/hpi_ts", zip_data) 632 | 633 | def hpi_ts_forecast(self, zip_data): 634 | """Call the hpi_ts_forecast endpoint""" 635 | return self.fetch_identifier_component("zip/hpi_ts_forecast", zip_data) 636 | 637 | def hpi_ts_historical(self, zip_data): 638 | """Call the hpi_ts_historical endpoint""" 639 | return self.fetch_identifier_component("zip/hpi_ts_historical", zip_data) 640 | 641 | def volatility(self, zip_data): 642 | """Call the volatility endpoint""" 643 | return self.fetch_identifier_component("zip/volatility", zip_data) 644 | 645 | def component_mget(self, zip_data, components): 646 | """Call the zip component_mget endpoint 647 | 648 | Args: 649 | - zip_data - As described in the class docstring. 650 | - components - A list of strings for each component to include in the request. 651 | Example: ["zip/details", "zip/volatility"] 652 | """ 653 | if not isinstance(components, list): 654 | print("Components param must be a list") 655 | return 656 | 657 | query_params = {"components": ",".join(components)} 658 | 659 | return self.fetch_identifier_component( 660 | "zip/component_mget", zip_data, query_params) 661 | 662 | 663 | class MsaComponentWrapper(ComponentWrapper): 664 | """MSA specific components 665 | 666 | All of the Analytics API msa endpoints take an ``msa_data`` argument. 667 | msa_data can be in the following forms: 668 | 669 | - A dict with an ``msa`` like: 670 | {"msa": "41860", "meta": "someId"} 671 | 672 | - A list of dicts as specified above: 673 | [{"msa": "41860", "meta": "someId"}, {"msa": "40928", "meta": "someId2}] 674 | 675 | - A single string representing a ``msa``: 676 | "41860" 677 | 678 | - A list of ``msa`` strings: 679 | ["41860", "40928"] 680 | 681 | The "meta" field is always optional. 682 | 683 | All of the msa endpoint methods return an MsaResponse, 684 | or the output of a custom OutputGenerator if one was specified in the constructor. 685 | """ 686 | 687 | def _convert_to_identifier_json(self, msa_data): 688 | if isinstance(msa_data, str): 689 | # allow just passing a msa string. 690 | return {"msa": msa_data} 691 | 692 | if isinstance(msa_data, dict): 693 | allowed_keys = ["msa", "meta"] 694 | 695 | # ensure the dict does not contain any unallowed keys 696 | for key in msa_data: 697 | if key not in allowed_keys: 698 | msg = "Key in msa input not allowed: " + key 699 | raise housecanary.exceptions.InvalidInputException(msg) 700 | 701 | # ensure it contains a "msa" key 702 | if "msa" in msa_data: 703 | return msa_data 704 | 705 | # if we made it here, the input was not valid. 706 | msg = ("Input is invalid. Must be a dict or list of dicts" 707 | " with each item containing at least 'msa' key.") 708 | raise housecanary.exceptions.InvalidInputException((msg)) 709 | 710 | def details(self, msa_data): 711 | """Call the details endpoint""" 712 | return self.fetch_identifier_component("msa/details", msa_data) 713 | 714 | def hpi_ts(self, msa_data): 715 | """Call the hpi_ts endpoint""" 716 | return self.fetch_identifier_component("msa/hpi_ts", msa_data) 717 | 718 | def hpi_ts_forecast(self, msa_data): 719 | """Call the hpi_ts_forecast endpoint""" 720 | return self.fetch_identifier_component("msa/hpi_ts_forecast", msa_data) 721 | 722 | def hpi_ts_historical(self, msa_data): 723 | """Call the hpi_ts_historical endpoint""" 724 | return self.fetch_identifier_component("msa/hpi_ts_historical", msa_data) 725 | 726 | def component_mget(self, msa_data, components): 727 | """Call the msa component_mget endpoint 728 | 729 | Args: 730 | - msa_data - As described in the class docstring. 731 | - components - A list of strings for each component to include in the request. 732 | Example: ["msa/details", "msa/hpi_ts"] 733 | """ 734 | if not isinstance(components, list): 735 | print("Components param must be a list") 736 | return 737 | 738 | query_params = {"components": ",".join(components)} 739 | 740 | return self.fetch_identifier_component( 741 | "msa/component_mget", msa_data, query_params) 742 | -------------------------------------------------------------------------------- /housecanary/authentication.py: -------------------------------------------------------------------------------- 1 | """ 2 | This module provides an authentication implementation for requests to the HouseCanary API. 3 | """ 4 | 5 | import hmac 6 | import hashlib 7 | import datetime 8 | import calendar 9 | from future import standard_library 10 | from urllib.parse import urlencode 11 | from urllib.parse import urlparse, parse_qsl, urlunparse 12 | from builtins import str 13 | import requests 14 | 15 | standard_library.install_aliases() 16 | 17 | 18 | class HCAuth(requests.auth.AuthBase): 19 | """ HouseCanary Authentication handler for Requests.""" 20 | 21 | def __init__(self, auth_key, auth_secret): 22 | """Create an authentication handler for HouseCanary API V1 requests 23 | 24 | Args: 25 | auth_key (string) - The HouseCanary API auth key 26 | auth_secret (string) - The HouseCanary API secret 27 | """ 28 | 29 | self._auth_key = auth_key 30 | self._auth_secret = auth_secret 31 | 32 | def __call__(self, request): 33 | # Override in subclass 34 | return request 35 | 36 | 37 | class HCAuthV1(HCAuth): 38 | """ HouseCanary API V1 Authentication handler for Requests. 39 | 40 | Generates an HMAC-SHA1 signature for the X-Auth-Signature header. 41 | """ 42 | 43 | SIGN_DELIMITER = "\n" 44 | AUTH_PROTO_V1 = "hc_hmac_v1" 45 | 46 | def __call__(self, request): 47 | # parse the url of the request 48 | scheme, netloc, path, params, query, fragment = urlparse(request.url) 49 | 50 | # convert the query string to a dict 51 | query_params = dict(parse_qsl(query)) 52 | 53 | # add additional query params that are required for authentication 54 | query_params['AuthKey'] = self._auth_key 55 | query_params['AuthProto'] = self.AUTH_PROTO_V1 56 | query_params['AuthTimestamp'] = str(self._get_timestamp_utc()) 57 | 58 | # back to string 59 | query_string = urlencode(query_params, True) 60 | 61 | # get the post data from the request, if any 62 | data = request.body or "" 63 | 64 | # set the auth signature as required by the HouseCanary API 65 | signature = self._get_signature(path, query_string, request.method, data) 66 | request.headers["X-Auth-Signature"] = signature 67 | 68 | # recreate the url with the updated query string 69 | updated_url = urlunparse([scheme, netloc, path, params, query_string, fragment]) 70 | request.url = requests.utils.requote_uri(updated_url) 71 | 72 | return request 73 | 74 | def _get_signature(self, endpoint_path, query_string, http_method, data): 75 | # all the inputs should be strings 76 | 77 | # uses hmac to produce a signature from the api secret and a message. 78 | # The message is defined as: 79 | # (HTTP_METHOD, HTTP_LOCATION, HTTP_QUERY_STRING, HTTP_POST_BODY) 80 | # concatenated by newline "\n" (unix style). 81 | sign_str = self.SIGN_DELIMITER.join([http_method, endpoint_path, query_string, data]) 82 | signature = hmac.new(str(self._auth_secret), sign_str, digestmod=hashlib.sha1).hexdigest() 83 | return signature 84 | 85 | @staticmethod 86 | def _get_timestamp_utc(): 87 | cur_utc_time = calendar.timegm(datetime.datetime.utcnow().utctimetuple()) 88 | return cur_utc_time 89 | -------------------------------------------------------------------------------- /housecanary/constants.py: -------------------------------------------------------------------------------- 1 | HTTP_CODE_OK = 200 2 | HTTP_FORBIDDEN = 403 3 | HTTP_TOO_MANY_REQUESTS = 429 4 | 5 | BIZ_CODE_OK = 0 6 | 7 | URL_PREFIX = "https://api.housecanary.com" 8 | DEFAULT_VERSION = "v2" 9 | -------------------------------------------------------------------------------- /housecanary/excel/__init__.py: -------------------------------------------------------------------------------- 1 | """Module for creating Excel exports of HouseCanary API data""" 2 | 3 | from __future__ import print_function 4 | import os 5 | import csv 6 | import time 7 | import io 8 | import sys 9 | import openpyxl 10 | from builtins import str 11 | from slugify import slugify 12 | from . import analytics_data_excel 13 | from . import utilities 14 | from .. import ApiClient 15 | from .. import exceptions 16 | 17 | 18 | def export_analytics_data_to_excel(data, output_file_name, result_info_key, identifier_keys): 19 | """Creates an Excel file containing data returned by the Analytics API 20 | 21 | Args: 22 | data: Analytics API data as a list of dicts 23 | output_file_name: File name for output Excel file (use .xlsx extension). 24 | 25 | """ 26 | workbook = create_excel_workbook(data, result_info_key, identifier_keys) 27 | workbook.save(output_file_name) 28 | print('Saved Excel file to {}'.format(output_file_name)) 29 | 30 | 31 | def export_analytics_data_to_csv(data, output_folder, result_info_key, identifier_keys): 32 | """Creates CSV files containing data returned by the Analytics API. 33 | Creates one file per requested endpoint and saves it into the 34 | specified output_folder 35 | 36 | Args: 37 | data: Analytics API data as a list of dicts 38 | output_folder: Path to a folder to save the CSV files into 39 | """ 40 | workbook = create_excel_workbook(data, result_info_key, identifier_keys) 41 | 42 | suffix = '.csv' 43 | 44 | if not os.path.exists(output_folder): 45 | os.makedirs(output_folder) 46 | 47 | for worksheet in workbook.worksheets: 48 | file_name = utilities.convert_title_to_snake_case(worksheet.title) 49 | 50 | file_path = os.path.join(output_folder, file_name + suffix) 51 | 52 | mode = 'w' 53 | if sys.version_info[0] < 3: 54 | mode = 'wb' 55 | with io.open(file_path, mode) as output_file: 56 | csv_writer = csv.writer(output_file) 57 | for row in worksheet.rows: 58 | csv_writer.writerow([cell.value for cell in row]) 59 | 60 | print('Saved CSV files to {}'.format(output_folder)) 61 | 62 | 63 | def concat_excel_reports(addresses, output_file_name, endpoint, report_type, 64 | retry, api_key, api_secret, files_path): 65 | """Creates an Excel file made up of combining the Value Report or Rental Report Excel 66 | output for the provided addresses. 67 | 68 | Args: 69 | addresses: A list of (address, zipcode) tuples 70 | output_file_name: A file name for the Excel output 71 | endpoint: One of 'value_report' or 'rental_report' 72 | report_type: One of 'full' or 'summary' 73 | retry: optional boolean to retry if rate limit is reached 74 | api_key: optional API Key 75 | api_secret: optional API Secret 76 | files_path: Path to save individual files. If None, don't save files 77 | """ 78 | # create the master workbook to output 79 | master_workbook = openpyxl.Workbook() 80 | 81 | if api_key is not None and api_secret is not None: 82 | client = ApiClient(api_key, api_secret) 83 | else: 84 | client = ApiClient() 85 | 86 | errors = [] 87 | 88 | # for each address, call the API and load the xlsx content in a workbook. 89 | for index, addr in enumerate(addresses): 90 | print('Processing {}'.format(addr[0])) 91 | result = _get_excel_report( 92 | client, endpoint, addr[0], addr[1], report_type, retry) 93 | 94 | if not result['success']: 95 | print('Error retrieving report for {}'.format(addr[0])) 96 | print(result['content']) 97 | errors.append({'address': addr[0], 'message': result['content']}) 98 | continue 99 | 100 | orig_wb = openpyxl.load_workbook(filename=io.BytesIO(result['content'])) 101 | 102 | _save_individual_file(orig_wb, files_path, addr[0]) 103 | 104 | # for each worksheet for this address 105 | for sheet_name in orig_wb.get_sheet_names(): 106 | # if worksheet doesn't exist in master workbook, create it 107 | if sheet_name in master_workbook.get_sheet_names(): 108 | master_ws = master_workbook.get_sheet_by_name(sheet_name) 109 | else: 110 | master_ws = master_workbook.create_sheet(sheet_name) 111 | 112 | # get all the rows in the address worksheet 113 | orig_rows = orig_wb.get_sheet_by_name(sheet_name).rows 114 | 115 | if sheet_name == 'Summary' or sheet_name == 'Chart Data': 116 | _process_non_standard_sheet(master_ws, orig_rows, addr, index) 117 | continue 118 | 119 | _process_standard_sheet(master_ws, orig_rows, addr, index) 120 | 121 | # remove the first sheet which will be empty 122 | master_workbook.remove(master_workbook.worksheets[0]) 123 | 124 | # if any errors occurred, write them to an "Errors" worksheet 125 | if len(errors) > 0: 126 | errors_sheet = master_workbook.create_sheet('Errors') 127 | for error_idx, error in enumerate(errors): 128 | errors_sheet.cell(row=error_idx+1, column=1, value=error['address']) 129 | errors_sheet.cell(row=error_idx+1, column=2, value=error['message']) 130 | 131 | # save the master workbook to output_file_name 132 | adjust_column_width_workbook(master_workbook) 133 | output_file_path = os.path.join(files_path, output_file_name) 134 | master_workbook.save(output_file_path) 135 | print('Saved output to {}'.format(output_file_path)) 136 | 137 | 138 | def _process_standard_sheet(master_ws, orig_rows, addr, address_index): 139 | # if this is the first address, add headers for address and zipcode 140 | # in the first two columns of the first row of the master worksheet 141 | if address_index == 0: 142 | master_ws.cell(row=1, column=1, value='Address') 143 | master_ws.cell(row=1, column=2, value='Zipcode') 144 | 145 | # get the next row in the master worksheet to start writing to. 146 | # this actually sets the next row to the last row with values in it, 147 | # but that's good because the first row of the next address sheet 148 | # is skipped in order to skip the header. 149 | next_row_idx = 1 if address_index == 0 else master_ws.max_row 150 | 151 | # go through the rows from the address worksheet 152 | for orig_row_idx, orig_row in enumerate(orig_rows): 153 | if address_index > 0 and orig_row_idx == 0: 154 | # after the first address, skip the header rows 155 | continue 156 | # write the address and zipcode columns 157 | if orig_row_idx > 0: 158 | master_ws.cell(row=next_row_idx + orig_row_idx, column=1, value=addr[0]) 159 | master_ws.cell(row=next_row_idx + orig_row_idx, column=2, value=addr[1]) 160 | 161 | # copy over the address sheet's cells 162 | # starting at the row we left off at and two columns over 163 | _copy_row_to_worksheet(master_ws, orig_row, next_row_idx, orig_row_idx) 164 | 165 | 166 | def _process_non_standard_sheet(master_ws, orig_rows, addr, address_index): 167 | # for the Summary sheet, there are multiple rows with different data, 168 | # so we'll simply copy the rows as they are 169 | 170 | # first, let's get the next row to write to, 171 | # leaving a space between the data from the previous address 172 | next_row_idx = 1 if address_index == 0 else master_ws.max_row + 2 173 | 174 | # write the address and zipcode 175 | master_ws.cell(row=next_row_idx, column=1, value=addr[0]) 176 | master_ws.cell(row=next_row_idx, column=2, value=addr[1]) 177 | 178 | for orig_row_idx, orig_row in enumerate(orig_rows): 179 | # copy over the address sheet's cells 180 | # starting at the row we left off at and two columns over 181 | _copy_row_to_worksheet(master_ws, orig_row, next_row_idx, orig_row_idx) 182 | 183 | 184 | def _copy_row_to_worksheet(master_ws, orig_row, next_row_idx, orig_row_idx): 185 | for orig_cell_idx, orig_cell in enumerate(orig_row): 186 | master_ws.cell( 187 | row=next_row_idx + orig_row_idx, 188 | column=orig_cell_idx + 3, 189 | value=orig_cell.value 190 | ) 191 | 192 | 193 | def _get_excel_report(client, endpoint, address, zipcode, report_type, retry): 194 | if retry: 195 | while True: 196 | try: 197 | return _make_report_request(client, endpoint, address, zipcode, report_type) 198 | except exceptions.RateLimitException as e: 199 | rate_limit = e.rate_limits[0] 200 | utilities.print_rate_limit_error(rate_limit) 201 | 202 | if rate_limit['reset_in_seconds'] >= 300: 203 | # Rate limit will take more than 5 minutes to reset, so just fail 204 | return {'success': False, 'content': str(e)} 205 | 206 | print('Will retry once rate limit resets...') 207 | time.sleep(rate_limit['reset_in_seconds']) 208 | except exceptions.RequestException as e: 209 | return {'success': False, 'content': str(e)} 210 | 211 | # otherwise just try once. 212 | try: 213 | return _make_report_request(client, endpoint, address, zipcode, report_type) 214 | except exceptions.RequestException as e: 215 | return {'success': False, 'content': str(e)} 216 | 217 | 218 | def _make_report_request(client, endpoint, address, zipcode, report_type): 219 | if endpoint == 'rental_report': 220 | response = client.property.rental_report(address, zipcode, 'xlsx') 221 | else: 222 | response = client.property.value_report(address, zipcode, report_type, 'xlsx') 223 | return {'success': True, 'content': response.content} 224 | 225 | 226 | def _save_individual_file(workbook, files_path, addr): 227 | if not os.path.exists(files_path): 228 | os.makedirs(files_path) 229 | 230 | file_name = slugify('{}-{}'.format(addr, time.strftime('%Y-%m-%d_%H-%M-%S'))) 231 | file_path = os.path.join(files_path, '{}.xlsx'.format(file_name)) 232 | 233 | workbook.save(file_path) 234 | print('Saved output to {}'.format(file_path)) 235 | 236 | 237 | def create_excel_workbook(data, result_info_key, identifier_keys): 238 | """Calls the analytics_data_excel module to create the Workbook""" 239 | workbook = analytics_data_excel.get_excel_workbook(data, result_info_key, identifier_keys) 240 | adjust_column_width_workbook(workbook) 241 | return workbook 242 | 243 | 244 | def adjust_column_width_workbook(workbook): 245 | """Adjust column width for all sheets in workbook.""" 246 | for worksheet in workbook.worksheets: 247 | adjust_column_width(worksheet) 248 | 249 | 250 | def adjust_column_width(worksheet): 251 | """Adjust column width in worksheet. 252 | 253 | Args: 254 | worksheet: worksheet to be adjusted 255 | """ 256 | dims = {} 257 | padding = 1 258 | for row in worksheet.rows: 259 | for cell in row: 260 | if not cell.value: 261 | continue 262 | dims[cell.column] = max( 263 | dims.get(cell.column, 0), 264 | len(str(cell.value)) 265 | ) 266 | for col, value in list(dims.items()): 267 | worksheet.column_dimensions[col].width = value + padding 268 | -------------------------------------------------------------------------------- /housecanary/excel/analytics_data_excel.py: -------------------------------------------------------------------------------- 1 | import copy 2 | import openpyxl 3 | 4 | from . import utilities 5 | 6 | 7 | KEY_TO_WORKSHEET_MAP = { 8 | 'Msa Details': 'MSA Details' 9 | } 10 | 11 | LEADING_COLUMNS = ( 12 | 'address', 13 | 'unit', 14 | 'city', 15 | 'state', 16 | 'zipcode', 17 | 'slug', 18 | 'block_id', 19 | 'msa', 20 | 'num_bins', 21 | 'property_type', 22 | 'client_value', 23 | 'client_value_sqft', 24 | 'meta' 25 | ) 26 | 27 | LEADING_WORKSHEETS = () 28 | 29 | 30 | def get_excel_workbook(api_data, result_info_key, identifier_keys): 31 | """Generates an Excel workbook object given api_data returned by the Analytics API 32 | 33 | Args: 34 | api_data: Analytics API data as a list of dicts (one per identifier) 35 | result_info_key: the key in api_data dicts that contains the data results 36 | identifier_keys: the list of keys used as requested identifiers 37 | (address, zipcode, block_id, etc) 38 | 39 | Returns: 40 | raw excel file data 41 | """ 42 | 43 | cleaned_data = [] 44 | 45 | for item_data in api_data: 46 | result_info = item_data.pop(result_info_key, {}) 47 | 48 | cleaned_item_data = {} 49 | 50 | if 'meta' in item_data: 51 | meta = item_data.pop('meta') 52 | cleaned_item_data['meta'] = meta 53 | 54 | for key in item_data: 55 | cleaned_item_data[key] = item_data[key]['result'] 56 | 57 | cleaned_item_data[result_info_key] = result_info 58 | 59 | cleaned_data.append(cleaned_item_data) 60 | 61 | data_list = copy.deepcopy(cleaned_data) 62 | 63 | workbook = openpyxl.Workbook() 64 | 65 | write_worksheets(workbook, data_list, result_info_key, identifier_keys) 66 | 67 | return workbook 68 | 69 | 70 | def write_worksheets(workbook, data_list, result_info_key, identifier_keys): 71 | """Writes rest of the worksheets to workbook. 72 | 73 | Args: 74 | workbook: workbook to write into 75 | data_list: Analytics API data as a list of dicts 76 | result_info_key: the key in api_data dicts that contains the data results 77 | identifier_keys: the list of keys used as requested identifiers 78 | (address, zipcode, block_id, etc) 79 | """ 80 | 81 | # we can use the first item to figure out the worksheet keys 82 | worksheet_keys = get_worksheet_keys(data_list[0], result_info_key) 83 | 84 | for key in worksheet_keys: 85 | 86 | title = key.split('/')[1] 87 | 88 | title = utilities.convert_snake_to_title_case(title) 89 | 90 | title = KEY_TO_WORKSHEET_MAP.get(title, title) 91 | 92 | if key == 'property/nod': 93 | # the property/nod endpoint needs to be split into two worksheets 94 | create_property_nod_worksheets(workbook, data_list, result_info_key, identifier_keys) 95 | else: 96 | # all other endpoints are written to a single worksheet 97 | 98 | # Maximum 31 characters allowed in sheet title 99 | worksheet = workbook.create_sheet(title=title[:31]) 100 | 101 | processed_data = process_data(key, data_list, result_info_key, identifier_keys) 102 | 103 | write_data(worksheet, processed_data) 104 | 105 | # remove the first, unused empty sheet 106 | workbook.remove_sheet(workbook.active) 107 | 108 | 109 | def create_property_nod_worksheets(workbook, data_list, result_info_key, identifier_keys): 110 | """Creates two worksheets out of the property/nod data because the data 111 | doesn't come flat enough to make sense on one sheet. 112 | 113 | Args: 114 | workbook: the main workbook to add the sheets to 115 | data_list: the main list of data 116 | result_info_key: the key in api_data dicts that contains the data results 117 | Should always be 'address_info' for property/nod 118 | identifier_keys: the list of keys used as requested identifiers 119 | (address, zipcode, city, state, etc) 120 | """ 121 | nod_details_list = [] 122 | nod_default_history_list = [] 123 | 124 | for prop_data in data_list: 125 | nod_data = prop_data['property/nod'] 126 | 127 | if nod_data is None: 128 | nod_data = {} 129 | 130 | default_history_data = nod_data.pop('default_history', []) 131 | 132 | _set_identifier_fields(nod_data, prop_data, result_info_key, identifier_keys) 133 | 134 | nod_details_list.append(nod_data) 135 | 136 | for item in default_history_data: 137 | _set_identifier_fields(item, prop_data, result_info_key, identifier_keys) 138 | nod_default_history_list.append(item) 139 | 140 | worksheet = workbook.create_sheet(title='NOD Details') 141 | write_data(worksheet, nod_details_list) 142 | 143 | worksheet = workbook.create_sheet(title='NOD Default History') 144 | write_data(worksheet, nod_default_history_list) 145 | 146 | 147 | def get_worksheet_keys(data_dict, result_info_key): 148 | """Gets sorted keys from the dict, ignoring result_info_key and 'meta' key 149 | Args: 150 | data_dict: dict to pull keys from 151 | 152 | Returns: 153 | list of keys in the dict other than the result_info_key 154 | """ 155 | keys = set(data_dict.keys()) 156 | keys.remove(result_info_key) 157 | if 'meta' in keys: 158 | keys.remove('meta') 159 | return sorted(keys) 160 | 161 | 162 | def get_keys(data_list, leading_columns=LEADING_COLUMNS): 163 | """Gets all possible keys from a list of dicts, sorting by leading_columns first 164 | 165 | Args: 166 | data_list: list of dicts to pull keys from 167 | leading_columns: list of keys to put first in the result 168 | 169 | Returns: 170 | list of keys to be included as columns in excel worksheet 171 | """ 172 | all_keys = set().union(*(list(d.keys()) for d in data_list)) 173 | 174 | leading_keys = [] 175 | 176 | for key in leading_columns: 177 | if key not in all_keys: 178 | continue 179 | leading_keys.append(key) 180 | all_keys.remove(key) 181 | 182 | return leading_keys + sorted(all_keys) 183 | 184 | 185 | def write_data(worksheet, data): 186 | """Writes data into worksheet. 187 | 188 | Args: 189 | worksheet: worksheet to write into 190 | data: data to be written 191 | """ 192 | if not data: 193 | return 194 | 195 | if isinstance(data, list): 196 | rows = data 197 | else: 198 | rows = [data] 199 | 200 | if isinstance(rows[0], dict): 201 | keys = get_keys(rows) 202 | worksheet.append([utilities.convert_snake_to_title_case(key) for key in keys]) 203 | for row in rows: 204 | values = [get_value_from_row(row, key) for key in keys] 205 | worksheet.append(values) 206 | elif isinstance(rows[0], list): 207 | for row in rows: 208 | values = [utilities.normalize_cell_value(value) for value in row] 209 | worksheet.append(values) 210 | else: 211 | for row in rows: 212 | worksheet.append([utilities.normalize_cell_value(row)]) 213 | 214 | 215 | def get_value_from_row(row, key): 216 | """Gets a value and normalizes it's value""" 217 | if key in row: 218 | return utilities.normalize_cell_value(row[key]) 219 | return '' 220 | 221 | 222 | def process_data(key, data_list, result_info_key, identifier_keys): 223 | """ Given a key as the endpoint name, pulls the data for that endpoint out 224 | of the data_list for each address, processes the data into a more 225 | excel-friendly format and returns that data. 226 | 227 | Args: 228 | key: the endpoint name of the data to process 229 | data_list: the main data list to take the data from 230 | result_info_key: the key in api_data dicts that contains the data results 231 | identifier_keys: the list of keys used as requested identifiers 232 | (address, zipcode, block_id, etc) 233 | 234 | Returns: 235 | A list of dicts (rows) to be written to a worksheet 236 | """ 237 | master_data = [] 238 | 239 | for item_data in data_list: 240 | data = item_data[key] 241 | 242 | if data is None: 243 | current_item_data = {} 244 | else: 245 | if key == 'property/value': 246 | current_item_data = data['value'] 247 | 248 | elif key == 'property/details': 249 | top_level_keys = ['property', 'assessment'] 250 | current_item_data = flatten_top_level_keys(data, top_level_keys) 251 | 252 | elif key == 'property/school': 253 | current_item_data = data['school'] 254 | 255 | school_list = [] 256 | for school_type_key in current_item_data: 257 | schools = current_item_data[school_type_key] 258 | for school in schools: 259 | school['school_type'] = school_type_key 260 | school['school_address'] = school['address'] 261 | school['school_zipcode'] = school['zipcode'] 262 | school_list.append(school) 263 | 264 | current_item_data = school_list 265 | 266 | elif key == 'property/value_forecast': 267 | current_item_data = {} 268 | for month_key in data: 269 | current_item_data[month_key] = data[month_key]['value'] 270 | 271 | elif key in ['property/value_within_block', 'property/rental_value_within_block']: 272 | current_item_data = flatten_top_level_keys(data, [ 273 | 'housecanary_value_percentile_range', 274 | 'housecanary_value_sqft_percentile_range', 275 | 'client_value_percentile_range', 276 | 'client_value_sqft_percentile_range' 277 | ]) 278 | 279 | elif key in ['property/zip_details', 'zip/details']: 280 | top_level_keys = ['multi_family', 'single_family'] 281 | current_item_data = flatten_top_level_keys(data, top_level_keys) 282 | 283 | else: 284 | current_item_data = data 285 | 286 | if isinstance(current_item_data, dict): 287 | _set_identifier_fields(current_item_data, item_data, result_info_key, identifier_keys) 288 | 289 | master_data.append(current_item_data) 290 | else: 291 | # it's a list 292 | for item in current_item_data: 293 | _set_identifier_fields(item, item_data, result_info_key, identifier_keys) 294 | 295 | master_data.extend(current_item_data) 296 | 297 | return master_data 298 | 299 | 300 | def _set_identifier_fields(item, item_data, result_info_key, identifier_keys): 301 | result_info = item_data[result_info_key] 302 | for k in identifier_keys: 303 | if k == 'meta': 304 | item['meta'] = item_data['meta'] 305 | 306 | # skip the special query param keys since they are not always returned in the result info 307 | elif k not in ['client_value', 'client_value_sqft', 'num_bins', 'property_type']: 308 | item[k] = result_info[k] 309 | 310 | 311 | def flatten_top_level_keys(data, top_level_keys): 312 | """ Helper method to flatten a nested dict of dicts (one level) 313 | 314 | Example: 315 | {'a': {'b': 'bbb'}} becomes {'a_-_b': 'bbb'} 316 | 317 | The separator '_-_' gets formatted later for the column headers 318 | 319 | Args: 320 | data: the dict to flatten 321 | top_level_keys: a list of the top level keys to flatten ('a' in the example above) 322 | """ 323 | flattened_data = {} 324 | 325 | for top_level_key in top_level_keys: 326 | if data[top_level_key] is None: 327 | flattened_data[top_level_key] = None 328 | else: 329 | for key in data[top_level_key]: 330 | flattened_data['{}_-_{}'.format(top_level_key, key)] = data[top_level_key][key] 331 | 332 | return flattened_data 333 | -------------------------------------------------------------------------------- /housecanary/excel/utilities.py: -------------------------------------------------------------------------------- 1 | """Utility functions for the housecanary.excel package""" 2 | 3 | from __future__ import print_function 4 | import json 5 | import csv 6 | import io 7 | import sys 8 | from builtins import map 9 | 10 | 11 | def normalize_cell_value(value): 12 | """Process value for writing into a cell. 13 | 14 | Args: 15 | value: any type of variable 16 | 17 | Returns: 18 | json serialized value if value is list or dict, else value 19 | """ 20 | if isinstance(value, dict) or isinstance(value, list): 21 | return json.dumps(value) 22 | return value 23 | 24 | 25 | def convert_snake_to_title_case(key): 26 | """Converts snake-cased key to title-cased key.""" 27 | return ' '.join(w.capitalize() for w in key.split('_')) 28 | 29 | 30 | def convert_title_to_snake_case(key): 31 | """Converts title-cased key to snake-cased key.""" 32 | return '_'.join(w.lower() for w in key.split(' ')) 33 | 34 | 35 | def get_addresses_from_input_file(input_file_name): 36 | """Read addresses from input file into list of tuples. 37 | This only supports address and zipcode headers 38 | """ 39 | mode = 'r' 40 | if sys.version_info[0] < 3: 41 | mode = 'rb' 42 | with io.open(input_file_name, mode) as input_file: 43 | reader = csv.reader(input_file, delimiter=',', quotechar='"') 44 | 45 | addresses = list(map(tuple, reader)) 46 | 47 | if len(addresses) == 0: 48 | raise Exception('No addresses found in input file') 49 | 50 | header_columns = list(column.lower() for column in addresses.pop(0)) 51 | 52 | try: 53 | address_index = header_columns.index('address') 54 | zipcode_index = header_columns.index('zipcode') 55 | except ValueError: 56 | raise Exception("""The first row of the input CSV must be a header that contains \ 57 | a column labeled 'address' and a column labeled 'zipcode'.""") 58 | 59 | return list((row[address_index], row[zipcode_index]) for row in addresses) 60 | 61 | 62 | def get_identifiers_from_input_file(input_file_name): 63 | """Read identifiers from input file into list of dicts with the header row values 64 | as keys, and the rest of the rows as values. 65 | """ 66 | valid_identifiers = ['address', 'zipcode', 'unit', 'city', 'state', 'slug', 'block_id', 'msa', 67 | 'num_bins', 'property_type', 'client_value', 'client_value_sqft', 'meta'] 68 | mode = 'r' 69 | if sys.version_info[0] < 3: 70 | mode = 'rb' 71 | with io.open(input_file_name, mode) as input_file: 72 | result = [{identifier: val for identifier, val in list(row.items()) 73 | if identifier in valid_identifiers} 74 | for row in csv.DictReader(input_file, skipinitialspace=True)] 75 | return result 76 | 77 | 78 | def print_rate_limit_error(rate_limit): 79 | print("You have hit the API rate limit") 80 | print("Rate limit period: ", rate_limit["period"]) 81 | print("Request limit: ", rate_limit["request_limit"]) 82 | print("Requests remaining: ", rate_limit["requests_remaining"]) 83 | print("Rate limit resets at: ", rate_limit["reset"]) 84 | print("Time until rate limit resets: ", rate_limit["time_to_reset"]) 85 | 86 | 87 | def get_all_endpoints(level): 88 | if level == 'property': 89 | return ['property/block_histogram_baths', 90 | 'property/block_histogram_beds', 91 | 'property/block_histogram_building_area', 92 | 'property/block_histogram_value', 93 | 'property/block_histogram_value_sqft', 94 | 'property/block_rental_value_distribution', 95 | 'property/block_value_distribution', 96 | 'property/block_value_ts', 97 | 'property/block_value_ts_historical', 98 | 'property/block_value_ts_forecast', 99 | 'property/census', 100 | 'property/details', 101 | 'property/flood', 102 | 'property/ltv', 103 | 'property/ltv_details', 104 | 'property/mortgage_lien', 105 | 'property/msa_details', 106 | 'property/msa_hpi_ts', 107 | 'property/msa_hpi_ts_forecast', 108 | 'property/msa_hpi_ts_historical', 109 | 'property/nod', 110 | 'property/owner_occupied', 111 | 'property/rental_value', 112 | 'property/rental_value_within_block', 113 | 'property/sales_history', 114 | 'property/school', 115 | 'property/value', 116 | 'property/value_forecast', 117 | 'property/value_within_block', 118 | 'property/zip_details', 119 | 'property/zip_hpi_forecast', 120 | 'property/zip_hpi_historical', 121 | 'property/zip_hpi_ts', 122 | 'property/zip_hpi_ts_forecast', 123 | 'property/zip_hpi_ts_historical', 124 | 'property/zip_volatility'] 125 | 126 | if level == 'block': 127 | return ['block/histogram_baths', 128 | 'block/histogram_beds', 129 | 'block/histogram_building_area', 130 | 'block/histogram_value', 131 | 'block/histogram_value_sqft', 132 | 'block/rental_value_distribution', 133 | 'block/value_distribution', 134 | 'block/value_ts', 135 | 'block/value_ts_forecast', 136 | 'block/value_ts_historical'] 137 | 138 | if level == 'zip': 139 | return ['zip/details', 140 | 'zip/hpi_forecast', 141 | 'zip/hpi_historical', 142 | 'zip/hpi_ts', 143 | 'zip/hpi_ts_forecast', 144 | 'zip/hpi_ts_historical', 145 | 'zip/volatility'] 146 | 147 | if level == 'msa': 148 | return ['msa/details', 149 | 'msa/hpi_ts', 150 | 'msa/hpi_ts_forecast', 151 | 'msa/hpi_ts_historical'] 152 | 153 | raise Exception('Invalid endpoint level specified: {}'.format(level)) 154 | -------------------------------------------------------------------------------- /housecanary/exceptions.py: -------------------------------------------------------------------------------- 1 | from . import utilities 2 | 3 | 4 | class RequestException(Exception): 5 | """Exception representing an error due to an incorrect request structure 6 | or missing required fields.""" 7 | 8 | def __init__(self, status_code, message): 9 | Exception.__init__(self) 10 | self._status_code = status_code 11 | self._message = message 12 | 13 | def __str__(self): 14 | return "%s (HTTP Status: %s)" % (self._message, self._status_code) 15 | 16 | 17 | class RateLimitException(RequestException): 18 | """Exception for 429 rate limit exceeded""" 19 | 20 | def __init__(self, status_code, message, response): 21 | RequestException.__init__(self, status_code, message) 22 | self._response = response 23 | self._rate_limits = None 24 | 25 | def __str__(self): 26 | return "%s (HTTP Status: %s) (Resets at: %s)" % ( 27 | self._message, self._status_code, self._get_rate_limit_reset()) 28 | 29 | @property 30 | def rate_limits(self): 31 | """Returns list of rate limit information from the response""" 32 | if not self._rate_limits: 33 | self._rate_limits = utilities.get_rate_limits(self._response) 34 | return self._rate_limits 35 | 36 | def _get_rate_limit_reset(self): 37 | return self.rate_limits[0]["reset"] 38 | 39 | 40 | class UnauthorizedException(RequestException): 41 | """Exception for unauthenticated or unauthorized request.""" 42 | pass 43 | 44 | 45 | class InvalidInputException(Exception): 46 | """Exception representing invalid input passed to the API Client.""" 47 | pass 48 | -------------------------------------------------------------------------------- /housecanary/hc_api_excel_concat/README.rst: -------------------------------------------------------------------------------- 1 | HouseCanary API Excel Concat 2 | ============================= 3 | 4 | HouseCanary API Excel Concat is a command line tool that allows you to call the 5 | Value Report or Rental Report API for multiple addresses by passing in a CSV file 6 | containing addresses and zip codes. 7 | 8 | The input CSV file must contain a header row with columns for ``address`` and ``zipcode``. 9 | Other columns can be included but will be ignored. 10 | See an example input `here <../../sample_input/sample-input.csv>`_. 11 | 12 | It generates a single .xlsx file which combines the Excel output of each address. 13 | 14 | Installation 15 | ------------ 16 | 17 | ``hc_api_excel_concat`` is installed as part of the HouseCanary client. 18 | If you haven't installed that yet, you can do so with ``pip``: 19 | 20 | :: 21 | 22 | pip install housecanary 23 | 24 | 25 | Usage instructions: 26 | ------------------- 27 | 28 | **Usage:** 29 | :: 30 | 31 | hc_api_excel_concat () [-o FILE] [-f PATH] [-e ENDPOINT] [-k KEY] [-s SECRET] [-t TYPE] [-h?] [-r] 32 | 33 | **Example:** 34 | :: 35 | 36 | hc_api_excel_concat sample-input.csv -o vr_output.xlsx -f vr_files -e value_report 37 | 38 | hc_api_excel_concat sample-input.csv -o rr_output.xlsx -f rr_files -e rental_report 39 | 40 | **Options:** 41 | 42 | - input 43 | 44 | Required. An input CSV file containing addresses and zipcodes 45 | 46 | - -o FILE --output=FILE 47 | 48 | Optional. A file name for the combined Excel output. The file name you specify should have the ``.xlsx`` extension. Defaults to ``output.xlsx`` 49 | 50 | - -f PATH --files=PATH 51 | 52 | Optional. A path to save the individual Excel files for each address. Defaults to 'output_files'. 53 | 54 | - -e ENDPOINT --endpoint=ENDPOINT 55 | 56 | Optional. One of 'value_report' or 'rental_report' to determine which API endpoint to call. Defaults to 'value_report' 57 | 58 | - -k KEY --key=KEY 59 | 60 | Optional API Key. Alternatively, you can use the HC_API_KEY environment variable 61 | 62 | - -s SECRET --secret=SECRET 63 | 64 | Optional API Secret. Alternatively, you can use the HC_API_SECRET environment variable 65 | 66 | - -t TYPE --type=TYPE 67 | 68 | Optional Report Type of ``full`` or ``summary``. Default is ``full`` 69 | 70 | - -r --retry 71 | 72 | Optional. When specified, if any of the API calls fail due to exceeding the rate limit, the command will wait and retry once the limit has reset. However, if the rate limit will take more than 5 minutes to reset, the retry flag is ignored and the command will exit. 73 | 74 | - -h -? --help 75 | 76 | Show usage instructions -------------------------------------------------------------------------------- /housecanary/hc_api_excel_concat/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/housecanary/hc-api-python/2bb9e2208b34e8617575de45934357ee33b8531c/housecanary/hc_api_excel_concat/__init__.py -------------------------------------------------------------------------------- /housecanary/hc_api_excel_concat/hc_api_excel_concat.py: -------------------------------------------------------------------------------- 1 | """hc_api_excel_concat - Takes a CSV file containing rows of addresses and zipcodes, 2 | calls the HouseCanary Value Report or Rental Report API to retrieve Excel output 3 | for the addresses and combines the results into a single Excel file. 4 | 5 | Usage: hc_api_excel_concat () [-o FILE] [-f PATH] [-e ENDPOINT] [-k KEY] [-s SECRET] [-t TYPE] [-h?] [-r] 6 | 7 | Examples: 8 | hc_api_excel_concat sample_input/sample-input.csv -o vr_output.xlsx -f output_files -e value_report 9 | 10 | hc_api_excel_concat sample_input/sample-input.csv -o rr_output.xlsx -f output_files -e rental_report 11 | 12 | Options: 13 | input Required. An input CSV file containing addresses and zipcodes 14 | 15 | -o FILE --output=FILE Optional. A file name for the Excel output. 16 | Defaults to 'output.xlsx' 17 | 18 | -f PATH --files=PATH Optional. A path to save the individual Excel files for each address. 19 | Defaults to 'output_files'. 20 | 21 | -e ENDPOINT --endpoint=ENDPOINT Optional. One of 'value_report' or 'rental_report' 22 | to determine which API endpoint to call. 23 | Defaults to 'value_report' 24 | 25 | -k KEY --key=KEY Optional API Key. Alternatively, you can use the HC_API_KEY 26 | environment variable 27 | 28 | -s SECRET --secret=SECRET Optional API Secret. Alternatively, you can use the HC_API_SECRET 29 | environment variable 30 | 31 | -t TYPE --type=TYPE Optional Report Type of 'full' or 'summary'. Default is 'full' 32 | 33 | -r --retry Optional. When specified, if any of the API calls fail due to 34 | exceeding the rate limit, the command will wait and retry once 35 | the limit has reset. However, if the rate limit will take more 36 | than 5 minutes to reset, the retry flag is ignored and the 37 | command will exit. 38 | 39 | -h -? --help Show usage 40 | """ 41 | 42 | 43 | from __future__ import print_function 44 | import sys 45 | from builtins import str 46 | from docopt import docopt 47 | import housecanary 48 | 49 | 50 | def hc_api_excel_concat(docopt_args): 51 | input_file_name = docopt_args[''] 52 | output_file_name = docopt_args['--output'] or 'output.xlsx' 53 | endpoint = docopt_args['--endpoint'] or 'value_report' 54 | api_key = docopt_args['--key'] or None 55 | api_secret = docopt_args['--secret'] or None 56 | report_type = docopt_args['--type'] or 'full' 57 | retry = docopt_args['--retry'] or False 58 | files_path = docopt_args['--files'] or 'output_files' 59 | 60 | try: 61 | addresses = housecanary.excel_utilities.get_addresses_from_input_file(input_file_name) 62 | except Exception as ex: 63 | print(str(ex)) 64 | sys.exit(2) 65 | 66 | if len(addresses) == 0: 67 | print('No addresses were found in the input file') 68 | sys.exit(2) 69 | 70 | if endpoint != 'value_report' and endpoint != 'rental_report': 71 | print("""Invalid endpoint '{}'. Must be one of 'value_report' or 'rental_report'. 72 | You can omit the endpoint param to default to 'value_report'""".format(endpoint)) 73 | sys.exit(2) 74 | 75 | housecanary.concat_excel_reports( 76 | addresses, output_file_name, endpoint, report_type, retry, api_key, api_secret, files_path) 77 | 78 | 79 | def main(): 80 | args = docopt(__doc__) 81 | hc_api_excel_concat(args) 82 | -------------------------------------------------------------------------------- /housecanary/hc_api_export/README.rst: -------------------------------------------------------------------------------- 1 | HouseCanary Analytics API Export 2 | ============================= 3 | 4 | HouseCanary Analytics API Export is a command line tool that allows you to call API endpoints 5 | with a CSV file containing properties, blocks, zip codes and MSAs. 6 | 7 | The input CSV file must contain a header row with columns indicating the identifiers. 8 | Allowed identifiers are: 9 | 10 | - **address** 11 | - **zipcode** 12 | - **unit** 13 | - **city** 14 | - **state** 15 | - **slug** 16 | - **block_id** 17 | - **msa** 18 | - **client_value** 19 | - **client_value_sqft** 20 | - **num_bins** 21 | - **property_type** 22 | - **meta** 23 | 24 | Other columns can be included but will be ignored. 25 | See some example inputs `here <../../sample_input/>`_. 26 | 27 | It generates an export of the Analytics API data in Excel or CSV format. 28 | 29 | If exporting to Excel, this creates a single Excel file with a worksheet per endpoint. 30 | 31 | If exporting to CSV, this creates a single CSV file per endpoint. 32 | 33 | Installation 34 | ------------ 35 | 36 | ``hc_api_export`` is installed as part of the HouseCanary client. If you haven't installed that yet, you can do so with ``pip``: 37 | 38 | :: 39 | 40 | pip install housecanary 41 | 42 | Usage instructions 43 | ------------------ 44 | 45 | **Usage:** 46 | 47 | :: 48 | 49 | hc_api_export ( ) [-t TYPE] [-o FILE] [-p PATH] [-k KEY] [-s SECRET] [-h?] [-r] 50 | 51 | **Examples:** 52 | 53 | :: 54 | 55 | hc_api_export sample-input.csv property/* -t excel -o output.xlsx 56 | 57 | hc_api_export sample-input.csv property/value,property/school -t csv -p /home/my_output 58 | 59 | hc_api_export sample-input-blocks.csv block/* -t excel -o block_output.xlsx 60 | 61 | hc_api_export sample-input-zipcodes.csv zip/* -t excel -o zip_output.xlsx 62 | 63 | hc_api_export sample-input-msas.csv msa/* -t excel -o msa_output.xlsx 64 | 65 | **Options:** 66 | 67 | - input 68 | 69 | Required. An input CSV file containing property, zipcode, block or MSA identifiers 70 | 71 | - endpoints 72 | 73 | Required. A comma separated list of endpoints to call like: ``property/value,property/school`` 74 | 75 | To call all property endpoints, use ``property/\*``. The same applies for `block`, `zipcode` and `msa` endpoints. 76 | 77 | Only one level of endpoints can be called at a time, meaning you can't mix `property` and `block` endpoints. 78 | 79 | - -t TYPE --type=TYPE 80 | 81 | Optional. An output type of ``excel`` or ``csv``. Default is ``excel`` 82 | 83 | - -o FILE --output=FILE 84 | 85 | Optional. A file name to output Excel results to. Only used when -t is ``excel``. Defaults to ``housecanary_output.xlsx`` 86 | 87 | - -p PATH --path=PATH 88 | 89 | Optional. A path to output CSV files to. Only used when -t is ``csv``. Defaults to ``housecanary_csv`` 90 | 91 | - -k KEY --key=KEY 92 | 93 | Optional API Key. Alternatively, you can use the HC_API_KEY environment variable 94 | 95 | - -s SECRET --secret=SECRET 96 | 97 | Optional API Secret. Alternatively, you can use the HC_API_SECRET environment variable 98 | 99 | - -r --retry 100 | 101 | Optional. When specified, if the API call fails due to exceeding the rate limit, the command will wait and retry once the limit has reset. However, if the rate limit will take more than 5 minutes to reset, the retry flag is ignored and the command will exit. 102 | 103 | - -h -? --help 104 | 105 | Show usage instructions -------------------------------------------------------------------------------- /housecanary/hc_api_export/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/housecanary/hc-api-python/2bb9e2208b34e8617575de45934357ee33b8531c/housecanary/hc_api_export/__init__.py -------------------------------------------------------------------------------- /housecanary/hc_api_export/hc_api_export.py: -------------------------------------------------------------------------------- 1 | """hc_api_export - Takes a CSV file containing rows of property, zipcode, block or MSA identifiers, 2 | calls the specified HouseCanary API endpoints to retrieve data 3 | for the identifiers and outputs the data to Excel or CSV. 4 | 5 | The input CSV file must contain a header row with columns indicating the identifiers. 6 | Allowed identifiers are: 7 | - address 8 | - zipcode 9 | - unit 10 | - city 11 | - state 12 | - slug 13 | - block_id 14 | - msa 15 | - client_value 16 | - client_value_sqft 17 | - num_bins 18 | - property_type 19 | - meta 20 | 21 | If exporting to Excel, this creates a single Excel file 22 | with a worksheet per endpoint. 23 | 24 | If exporting to CSV, this creates a single CSV file per endpoint. 25 | 26 | Usage: hc_api_export ( ) [-t TYPE] [-o FILE] [-p PATH] [-k KEY] [-s SECRET] [-h?] [-r] 27 | 28 | Examples: 29 | hc_api_export sample_input/sample-input.csv property/* -t excel -o output.xlsx 30 | 31 | hc_api_export sample_input/sample-input.csv property/value,property/school -t csv -p /home/my_output 32 | 33 | hc_api_export sample_input/sample-input-blocks.csv block/* -t excel -o block_output.xlsx 34 | 35 | hc_api_export sample_input/sample-input-zipcodes.csv zip/* -t excel -o zip_output.xlsx 36 | 37 | hc_api_export sample_input/sample-input-msas.csv msa/* -t excel -o msa_output.xlsx 38 | 39 | Options: 40 | input Required. An input CSV file containing addresses and zipcodes 41 | 42 | endpoints Required. A comma separated list of endpoints to call like: 43 | 'property/value,property/school' 44 | To call all endpoints, 45 | use 'property/*', 'block/*', 'zipcode/*' or 'msa/*'. 46 | Only one level of endpoints can be called at a time, 47 | meaning you can't mix 'property' and 'block' endpoints. 48 | 49 | -t TYPE --type=TYPE Optional. An output type of 'excel' or 'csv'. Default is 'excel' 50 | 51 | -o FILE --output=FILE Optional. A file name to output Excel results to. 52 | Only used when -t is 'excel'. 53 | Defaults to 'housecanary_output.xlsx' 54 | 55 | -p PATH --path=PATH Optional. A path to output CSV files to. 56 | Only used when -t is 'csv'. 57 | Defaults to 'housecanary_csv' 58 | 59 | -k KEY --key=KEY Optional API Key. Alternatively, you can use the HC_API_KEY 60 | environment variable 61 | 62 | -s SECRET --secret=SECRET Optional API Secret. Alternatively, you can use the HC_API_SECRET 63 | environment variable 64 | 65 | -r --retry Optional. When specified, if the API call fails due to exceeding 66 | the rate limit, the command will wait and retry once the limit 67 | has reset. However, if the rate limit will take more than 5 minutes 68 | to reset, the retry flag is ignored and the command will exit. 69 | 70 | -h -? --help Show usage 71 | """ 72 | 73 | 74 | from __future__ import print_function 75 | import sys 76 | import time 77 | from builtins import str 78 | from docopt import docopt 79 | import housecanary 80 | 81 | 82 | def hc_api_export(docopt_args): 83 | input_file_name = docopt_args[''] 84 | output_type = docopt_args['--type'] or 'excel' 85 | output_file_name = docopt_args['--output'] or 'housecanary_output.xlsx' 86 | output_csv_path = docopt_args['--path'] or 'housecanary_csv' 87 | endpoints = docopt_args[''] 88 | api_key = docopt_args['--key'] or None 89 | api_secret = docopt_args['--secret'] or None 90 | retry = docopt_args['--retry'] or False 91 | 92 | try: 93 | identifiers = housecanary.excel_utilities.get_identifiers_from_input_file(input_file_name) 94 | except Exception as ex: 95 | print(str(ex)) 96 | sys.exit(2) 97 | 98 | if len(identifiers) == 0: 99 | print('No identifiers were found in the input file') 100 | sys.exit(2) 101 | 102 | if ',' in endpoints: 103 | endpoints = endpoints.split(',') 104 | elif '*' in endpoints: 105 | endpoints = housecanary.excel_utilities.get_all_endpoints(endpoints.split('/')[0]) 106 | else: 107 | endpoints = [endpoints] 108 | 109 | if 'property/value_report' in endpoints or 'property/rental_report' in endpoints: 110 | print(("property/value_report and property/rental_report" 111 | "are not allowed for export to Excel." 112 | "Please use the Value Report application to get Excel outputs of Value Reports.")) 113 | return 114 | 115 | if retry: 116 | # If rate limit exceeded, enter retry process 117 | api_result = __get_results_from_api_with_retry(identifiers, endpoints, api_key, api_secret) 118 | else: 119 | # Just try once and exit if rate limit exceeded 120 | try: 121 | api_result = _get_results_from_api(identifiers, endpoints, api_key, api_secret) 122 | except housecanary.exceptions.RateLimitException as e: 123 | housecanary.excel_utilities.print_rate_limit_error(e.rate_limits[0]) 124 | sys.exit(2) 125 | 126 | all_data = api_result.json() 127 | 128 | result_info_key = _get_result_info_key(endpoints[0].split('/')[0]) 129 | identifier_keys = list(identifiers[0].keys()) 130 | 131 | if output_type.lower() == 'csv': 132 | housecanary.export_analytics_data_to_csv( 133 | all_data, output_csv_path, result_info_key, identifier_keys) 134 | else: 135 | housecanary.export_analytics_data_to_excel( 136 | all_data, output_file_name, result_info_key, identifier_keys) 137 | 138 | 139 | def __get_results_from_api_with_retry(identifiers, endpoints, api_key, api_secret): 140 | while True: 141 | try: 142 | return _get_results_from_api(identifiers, endpoints, api_key, api_secret) 143 | except housecanary.exceptions.RateLimitException as e: 144 | rate_limit = e.rate_limits[0] 145 | housecanary.excel_utilities.print_rate_limit_error(rate_limit) 146 | if rate_limit["reset_in_seconds"] < 300: 147 | print("Will retry once rate limit resets...") 148 | time.sleep(rate_limit["reset_in_seconds"]) 149 | else: 150 | # Rate limit will take more than 5 minutes to reset, so just exit 151 | sys.exit(2) 152 | 153 | 154 | def _get_results_from_api(identifiers, endpoints, api_key, api_secret): 155 | """Use the HouseCanary API Python Client to access the API""" 156 | 157 | if api_key is not None and api_secret is not None: 158 | client = housecanary.ApiClient(api_key, api_secret) 159 | else: 160 | client = housecanary.ApiClient() 161 | 162 | wrapper = getattr(client, endpoints[0].split('/')[0]) 163 | 164 | if len(endpoints) > 1: 165 | # use component_mget to request multiple endpoints in one call 166 | return wrapper.component_mget(identifiers, endpoints) 167 | else: 168 | return wrapper.fetch_identifier_component(endpoints[0], identifiers) 169 | 170 | 171 | def _get_result_info_key(level): 172 | if level == 'property': 173 | return 'address_info' 174 | if level == 'block': 175 | return 'block_info' 176 | if level == 'zip': 177 | return 'zipcode_info' 178 | if level == 'msa': 179 | return 'msa_info' 180 | raise Exception('Invalid endpoint level found: {}'.format(level)) 181 | 182 | 183 | def main(): 184 | args = docopt(__doc__) 185 | hc_api_export(args) 186 | -------------------------------------------------------------------------------- /housecanary/object.py: -------------------------------------------------------------------------------- 1 | """ 2 | Provides a HouseCanaryObject class which encapsulates 3 | an object and its associated data. 4 | Currently, only the Property subclass is implemented. 5 | """ 6 | 7 | from builtins import str 8 | from builtins import next 9 | from builtins import object 10 | import housecanary.constants as constants 11 | 12 | 13 | def _create_component_results(json_data, result_key): 14 | """ Returns a list of ComponentResult from the json_data""" 15 | component_results = [] 16 | for key, value in list(json_data.items()): 17 | if key not in [result_key, "meta"]: 18 | component_result = ComponentResult( 19 | key, 20 | value["result"], 21 | value["api_code"], 22 | value["api_code_description"] 23 | ) 24 | 25 | component_results.append(component_result) 26 | 27 | return component_results 28 | 29 | 30 | class HouseCanaryObject(object): 31 | """Base class for returned API objects.""" 32 | 33 | def __init__(self): 34 | """ 35 | Args: 36 | data - Json data returned from the API for this object. 37 | api_code - The HouseCanary business logic error code. 38 | api_code_description - The HouseCanary business logic error description. 39 | """ 40 | self.component_results = [] 41 | 42 | def has_error(self): 43 | """Returns whether there was a business logic error when fetching data 44 | for any components for this property. 45 | 46 | Returns: 47 | boolean 48 | """ 49 | return next( 50 | (True for cr in self.component_results 51 | if cr.has_error()), 52 | False 53 | ) 54 | 55 | def get_errors(self): 56 | """If there were any business errors fetching data for this property, 57 | returns the error messages. 58 | 59 | Returns: 60 | string - the error message, or None if there was no error. 61 | 62 | """ 63 | return [{cr.component_name: cr.get_error()} 64 | for cr in self.component_results if cr.has_error()] 65 | 66 | def __str__(self): 67 | return "HouseCanaryObject" 68 | 69 | 70 | class Property(HouseCanaryObject): 71 | """A single address""" 72 | 73 | def __init__(self, address=None, zipcode=None): 74 | """ 75 | Args: 76 | address (required) -- Building number, street name and unit number. 77 | zipcode (required) -- Zipcode that matches the address. 78 | data (optional) -- The data returned from the API for this property. 79 | api_code (optional) -- The HouseCanary business logic 80 | error code reflecting any error with this property. 81 | api_code_description (optional) -- The HouseCanary business logic 82 | error description. 83 | """ 84 | super(Property, self).__init__() 85 | 86 | self.address = str(address) 87 | self.zipcode = str(zipcode) 88 | self.block_id = None 89 | self.zipcode_plus4 = None 90 | self.address_full = None 91 | self.city = None 92 | self.county_fips = None 93 | self.geo_precision = None 94 | self.lat = None 95 | self.lng = None 96 | self.slug = None 97 | self.state = None 98 | self.unit = None 99 | self.meta = None 100 | 101 | @classmethod 102 | def create_from_json(cls, json_data): 103 | """Deserialize property json data into a Property object 104 | 105 | Args: 106 | json_data (dict): The json data for this property 107 | 108 | Returns: 109 | Property object 110 | 111 | """ 112 | prop = Property() 113 | address_info = json_data["address_info"] 114 | prop.address = address_info["address"] 115 | prop.block_id = address_info["block_id"] 116 | prop.zipcode = address_info["zipcode"] 117 | prop.zipcode_plus4 = address_info["zipcode_plus4"] 118 | prop.address_full = address_info["address_full"] 119 | prop.city = address_info["city"] 120 | prop.county_fips = address_info["county_fips"] 121 | prop.geo_precision = address_info["geo_precision"] 122 | prop.lat = address_info["lat"] 123 | prop.lng = address_info["lng"] 124 | prop.slug = address_info["slug"] 125 | prop.state = address_info["state"] 126 | prop.unit = address_info["unit"] 127 | 128 | prop.meta = None 129 | if "meta" in json_data: 130 | prop.meta = json_data["meta"] 131 | 132 | prop.component_results = _create_component_results(json_data, "address_info") 133 | 134 | return prop 135 | 136 | def __str__(self): 137 | return self.address or self.meta or "PropertyObject" 138 | 139 | 140 | class Block(HouseCanaryObject): 141 | """A single block""" 142 | 143 | def __init__(self, block_id=None): 144 | """ 145 | Args: 146 | block_id (required) -- Block ID. 147 | data (optional) -- The data returned from the API for this block. 148 | api_code (optional) -- The HouseCanary business logic 149 | error code reflecting any error with this block. 150 | api_code_description (optional) -- The HouseCanary business logic error description. 151 | """ 152 | super(Block, self).__init__() 153 | 154 | self.block_id = str(block_id) 155 | self.num_bins = None 156 | self.property_type = None 157 | self.meta = None 158 | 159 | @classmethod 160 | def create_from_json(cls, json_data): 161 | """Deserialize block json data into a Block object 162 | 163 | Args: 164 | json_data (dict): The json data for this block 165 | 166 | Returns: 167 | Block object 168 | 169 | """ 170 | block = Block() 171 | block_info = json_data["block_info"] 172 | block.block_id = block_info["block_id"] 173 | block.num_bins = block_info["num_bins"] if "num_bins" in block_info else None 174 | block.property_type = block_info["property_type"] if "property_type" in block_info else None 175 | block.meta = json_data["meta"] if "meta" in json_data else None 176 | 177 | block.component_results = _create_component_results(json_data, "block_info") 178 | 179 | return block 180 | 181 | def __str__(self): 182 | return self.block_id or self.meta or "BlockObject" 183 | 184 | 185 | class ZipCode(HouseCanaryObject): 186 | """A single zipcode""" 187 | 188 | def __init__(self, zipcode=None): 189 | """ 190 | Args: 191 | zipcode (required) -- Zipcode. 192 | data (optional) -- The data returned from the API for this zipcode. 193 | api_code (optional) -- The HouseCanary business logic 194 | error code reflecting any error with this zipcode. 195 | api_code_description (optional) -- The HouseCanary business logic error description. 196 | """ 197 | super(ZipCode, self).__init__() 198 | 199 | self.zipcode = str(zipcode) 200 | self.meta = None 201 | 202 | @classmethod 203 | def create_from_json(cls, json_data): 204 | """Deserialize zipcode json data into a ZipCode object 205 | 206 | Args: 207 | json_data (dict): The json data for this zipcode 208 | 209 | Returns: 210 | Zip object 211 | 212 | """ 213 | zipcode = ZipCode() 214 | zipcode.zipcode = json_data["zipcode_info"]["zipcode"] 215 | zipcode.meta = json_data["meta"] if "meta" in json_data else None 216 | 217 | zipcode.component_results = _create_component_results(json_data, "zipcode_info") 218 | 219 | return zipcode 220 | 221 | def __str__(self): 222 | return self.zipcode or self.meta or "ZipCodeObject" 223 | 224 | 225 | class Msa(HouseCanaryObject): 226 | """A single MSA""" 227 | 228 | def __init__(self, msa=None): 229 | """ 230 | Args: 231 | msa (required) -- MSA. 232 | data (optional) -- The data returned from the API for this MSA. 233 | api_code (optional) -- The HouseCanary business logic 234 | error code reflecting any error with this MSA. 235 | api_code_description (optional) -- The HouseCanary business logic error description. 236 | """ 237 | super(Msa, self).__init__() 238 | 239 | self.msa = str(msa) 240 | self.meta = None 241 | 242 | @classmethod 243 | def create_from_json(cls, json_data): 244 | """Deserialize msa json data into a Msa object 245 | 246 | Args: 247 | json_data (dict): The json data for this msa 248 | 249 | Returns: 250 | Msa object 251 | 252 | """ 253 | msa = Msa() 254 | msa.msa = json_data["msa_info"]["msa"] 255 | msa.meta = json_data["meta"] if "meta" in json_data else None 256 | 257 | msa.component_results = _create_component_results(json_data, "msa_info") 258 | 259 | return msa 260 | 261 | def __str__(self): 262 | return self.msa or self.meta or "MsaObject" 263 | 264 | 265 | class ComponentResult(object): 266 | """The results of a single component""" 267 | 268 | def __init__(self, component_name, json_data, api_code, api_code_description): 269 | """ 270 | Args: 271 | component_name - string name of the component. 272 | json_data - Json data returned from the API for this object. 273 | api_code - The HouseCanary business logic error code. 274 | api_code_description - The HouseCanary business logic error description. 275 | """ 276 | self.component_name = component_name 277 | self.json_data = json_data 278 | self.api_code = api_code 279 | self.api_code_description = api_code_description 280 | 281 | def has_error(self): 282 | """Returns whether this component had a business logic error""" 283 | return self.api_code > constants.BIZ_CODE_OK 284 | 285 | def get_error(self): 286 | """Gets the error of this component, if any""" 287 | return self.api_code_description 288 | -------------------------------------------------------------------------------- /housecanary/output.py: -------------------------------------------------------------------------------- 1 | """ 2 | This module provides a base class and two implementations of OutputGenerator, 3 | which is responsible for processing a response before returning a result to the caller. 4 | 5 | An OutputGenerator must implement a `process_response` method 6 | which takes an input (usually a server's response to an HTTP request) 7 | and returns something useful to the caller. This is a good place to do 8 | custom serialization of an API response. 9 | """ 10 | 11 | try: 12 | # Python 3 13 | from urllib.parse import urlparse 14 | except ImportError: 15 | # Python 2 16 | from urlparse import urlparse 17 | 18 | from housecanary.response import Response 19 | import housecanary.exceptions 20 | import housecanary.constants as constants 21 | 22 | 23 | class OutputGenerator(object): 24 | """Base class of an OutputGenerator. This base class just returns the given response.""" 25 | 26 | def process_response(self, response): 27 | """Simply returns the response passed in.""" 28 | return response 29 | 30 | 31 | class JsonOutputGenerator(OutputGenerator): 32 | """Returns the JSON body of the response 33 | 34 | Expects the given response to implement a `json` method, 35 | like the response from the requests library. 36 | """ 37 | 38 | def process_response(self, response): 39 | """Return the result of calling the json method on response""" 40 | return response.json() 41 | 42 | 43 | class ResponseOutputGenerator(OutputGenerator): 44 | """Serializes the response into an instance of Response. 45 | 46 | Expects the given response to be a response from the requests library. 47 | """ 48 | 49 | def process_response(self, response): 50 | content_type = response.headers['content-type'] 51 | 52 | if content_type == "application/json": 53 | return self.process_json_response(response) 54 | elif content_type == "application/pdf": 55 | return self.process_pdf_response(response) 56 | elif content_type == "appliction/zip": 57 | return self.process_zip_response(response) 58 | else: 59 | return response 60 | 61 | def process_json_response(self, response): 62 | """For a json response, check if there was any error and throw exception. 63 | Otherwise, create a housecanary.response.Response.""" 64 | 65 | response_json = response.json() 66 | 67 | # handle errors 68 | code_key = "code" 69 | if code_key in response_json and response_json[code_key] != constants.HTTP_CODE_OK: 70 | code = response_json[code_key] 71 | 72 | message = response_json 73 | if "message" in response_json: 74 | message = response_json["message"] 75 | elif "code_description" in response_json: 76 | message = response_json["code_description"] 77 | 78 | if code == constants.HTTP_FORBIDDEN: 79 | raise housecanary.exceptions.UnauthorizedException(code, message) 80 | if code == constants.HTTP_TOO_MANY_REQUESTS: 81 | raise housecanary.exceptions.RateLimitException(code, message, response) 82 | else: 83 | raise housecanary.exceptions.RequestException(code, message) 84 | 85 | request_url = response.request.url 86 | 87 | endpoint_name = self._parse_endpoint_name_from_url(request_url) 88 | 89 | return Response.create(endpoint_name, response_json, response) 90 | 91 | def process_pdf_response(self, response): 92 | return response.text 93 | 94 | def process_zip_response(self, response): 95 | return response.text 96 | 97 | @staticmethod 98 | def _parse_endpoint_name_from_url(request_url): 99 | # get the path from the url 100 | path = urlparse(request_url).path 101 | 102 | # path is like "/v2/property/value" 103 | 104 | # strip off the leading "/" 105 | path = path[1:] 106 | 107 | # keep only the part after the version and "/" 108 | path = path[path.find("/")+1:] 109 | 110 | # path is now like "property/value" 111 | return path 112 | -------------------------------------------------------------------------------- /housecanary/requestclient.py: -------------------------------------------------------------------------------- 1 | """ 2 | Provides a base client for making API requests. 3 | """ 4 | 5 | from builtins import object 6 | import requests 7 | 8 | import housecanary 9 | 10 | USER_AGENT = 'hc-client-python/%s %s' % ( 11 | housecanary.__version__, requests.utils.default_user_agent() 12 | ) 13 | 14 | 15 | class RequestClient(object): 16 | """Base class for making http requests with the 'requests' lib.""" 17 | 18 | def __init__(self, output_generator=None, authenticator=None): 19 | """ 20 | Args: 21 | output_generator - Optional. An instance of an OutputGenerator that implements 22 | a `process_response` method. Can be used to implement custom 23 | serialization of the response data from the `execute_request` 24 | method. If not specified, `execute_request` returns the 25 | response from the requests lib unchanged. 26 | authenticator - Optional. An instance of a requests.auth.AuthBase implementation 27 | for providing authentication to the request. 28 | """ 29 | self._output_generator = output_generator 30 | self._auth = authenticator 31 | 32 | def execute_request(self, url, http_method, query_params, post_data): 33 | """Makes a request to the specified url endpoint with the 34 | specified http method, params and post data. 35 | 36 | Args: 37 | url (string): The url to the API without query params. 38 | Example: "https://api.housecanary.com/v2/property/value" 39 | http_method (string): The http method to use for the request. 40 | query_params (dict): Dictionary of query params to add to the request. 41 | post_data: Json post data to send in the body of the request. 42 | 43 | Returns: 44 | The result of calling this instance's OutputGenerator process_response method 45 | on the requests.Response object. 46 | If no OutputGenerator is specified for this instance, returns the requests.Response. 47 | """ 48 | 49 | response = requests.request(http_method, url, params=query_params, 50 | auth=self._auth, json=post_data, 51 | headers={'User-Agent': USER_AGENT}) 52 | 53 | if isinstance(self._output_generator, str) and self._output_generator.lower() == "json": 54 | # shortcut for just getting json back 55 | return response.json() 56 | elif self._output_generator is not None: 57 | return self._output_generator.process_response(response) 58 | else: 59 | return response 60 | 61 | def get(self, url, query_params): 62 | """Makes a GET request to the specified url endpoint. 63 | 64 | Args: 65 | url (string): The url to the API without query params. 66 | Example: "https://api.housecanary.com/v2/property/value" 67 | query_params (dict): Dictionary of query params to add to the request. 68 | 69 | Returns: 70 | The result of calling this instance's OutputGenerator process_response method 71 | on the requests.Response object. 72 | If no OutputGenerator is specified for this instance, returns the requests.Response. 73 | """ 74 | return self.execute_request(url, "GET", query_params, None) 75 | 76 | def post(self, url, post_data, query_params=None): 77 | """Makes a POST request to the specified url endpoint. 78 | 79 | Args: 80 | url (string): The url to the API without query params. 81 | Example: "https://api.housecanary.com/v2/property/value" 82 | post_data: Json post data to send in the body of the request. 83 | query_params (dict): Optional. Dictionary of query params to add to the request. 84 | 85 | Returns: 86 | The result of calling this instance's OutputGenerator process_response method 87 | on the requests.Response object. 88 | If no OutputGenerator is specified for this instance, returns the requests.Response. 89 | """ 90 | if query_params is None: 91 | query_params = {} 92 | 93 | return self.execute_request(url, "POST", query_params, post_data) 94 | -------------------------------------------------------------------------------- /housecanary/response.py: -------------------------------------------------------------------------------- 1 | """ 2 | Provides Response to encapsulate API responses. 3 | """ 4 | 5 | from builtins import next 6 | from builtins import str 7 | from builtins import object 8 | from housecanary.object import Property 9 | from housecanary.object import Block 10 | from housecanary.object import ZipCode 11 | from housecanary.object import Msa 12 | from . import utilities 13 | 14 | 15 | class Response(object): 16 | """Encapsulate an API reponse.""" 17 | 18 | def __init__(self, endpoint_name, json_body, original_response): 19 | """ 20 | Args: 21 | endpoint_name (str) - The endpoint of the request, such as "property/value" 22 | json_body - The response body in json format. 23 | original_response (response object) - server response returned from an http request. 24 | """ 25 | self._endpoint_name = endpoint_name 26 | self._json_body = json_body 27 | self._response = original_response 28 | self._objects = [] 29 | self._has_object_error = None 30 | self._object_errors = None 31 | self._rate_limits = None 32 | 33 | @classmethod 34 | def create(cls, endpoint_name, json_body, original_response): 35 | """Factory for creating the correct type of Response based on the data. 36 | Args: 37 | endpoint_name (str) - The endpoint of the request, such as "property/value" 38 | json_body - The response body in json format. 39 | original_response (response object) - server response returned from an http request. 40 | """ 41 | 42 | if endpoint_name == "property/value_report": 43 | return ValueReportResponse(endpoint_name, json_body, original_response) 44 | 45 | if endpoint_name == "property/rental_report": 46 | return RentalReportResponse(endpoint_name, json_body, original_response) 47 | 48 | prefix = endpoint_name.split("/")[0] 49 | 50 | if prefix == "block": 51 | return BlockResponse(endpoint_name, json_body, original_response) 52 | 53 | if prefix == "zip": 54 | return ZipCodeResponse(endpoint_name, json_body, original_response) 55 | 56 | if prefix == "msa": 57 | return MsaResponse(endpoint_name, json_body, original_response) 58 | 59 | return PropertyResponse(endpoint_name, json_body, original_response) 60 | 61 | @property 62 | def endpoint_name(self): 63 | """Get the component name of the original request. 64 | 65 | Returns: 66 | Component name as a string. 67 | """ 68 | return self._endpoint_name 69 | 70 | @property 71 | def response(self): 72 | """Gets the original response 73 | 74 | Returns: 75 | response object passed in during instantiation. 76 | """ 77 | return self._response 78 | 79 | def json(self): 80 | """Gets the response body as json 81 | 82 | Returns: 83 | Json of the response body 84 | """ 85 | return self._json_body 86 | 87 | def get_object_errors(self): 88 | """Gets a list of business error message strings 89 | for each of the requested objects that had a business error. 90 | If there was no error, returns an empty list 91 | 92 | Returns: 93 | List of strings 94 | """ 95 | if self._object_errors is None: 96 | self._object_errors = [{str(o): o.get_errors()} 97 | for o in self.objects() 98 | if o.has_error()] 99 | 100 | return self._object_errors 101 | 102 | def has_object_error(self): 103 | """Returns true if any requested object had a business logic error, 104 | otherwise returns false 105 | 106 | Returns: 107 | boolean 108 | """ 109 | if self._has_object_error is None: 110 | # scan the objects for any business error codes 111 | self._has_object_error = next( 112 | (True for o in self.objects() 113 | if o.has_error()), 114 | False) 115 | return self._has_object_error 116 | 117 | def objects(self): 118 | """Override in subclasses""" 119 | raise NotImplementedError() 120 | 121 | def _get_objects(self, obj_type): 122 | if not self._objects: 123 | body = self.json() 124 | 125 | self._objects = [] 126 | 127 | if not isinstance(body, list): 128 | # The endpoints return a list in the body. 129 | # This could maybe raise an exception. 130 | return [] 131 | 132 | for item in body: 133 | prop = obj_type.create_from_json(item) 134 | self._objects.append(prop) 135 | 136 | return self._objects 137 | 138 | @property 139 | def rate_limits(self): 140 | """Returns a list of rate limit details.""" 141 | if not self._rate_limits: 142 | self._rate_limits = utilities.get_rate_limits(self.response) 143 | 144 | return self._rate_limits 145 | 146 | 147 | class PropertyResponse(Response): 148 | """Represents the data returned from an Analytics API property endpoint.""" 149 | 150 | def objects(self): 151 | """Gets a list of Property objects for the requested properties, 152 | each containing the property's returned json data from the API. 153 | 154 | Returns an empty list if the request format was PDF. 155 | 156 | Returns: 157 | List of Property objects 158 | """ 159 | return self._get_objects(Property) 160 | 161 | def properties(self): 162 | """Alias method for objects.""" 163 | return self.objects() 164 | 165 | 166 | class BlockResponse(Response): 167 | """Represents the data returned from an Analytics API block endpoint.""" 168 | 169 | def objects(self): 170 | """Gets a list of Block objects for the requested blocks, 171 | each containing the block's returned json data from the API. 172 | 173 | Returns: 174 | List of Block objects 175 | """ 176 | return self._get_objects(Block) 177 | 178 | def blocks(self): 179 | """Alias method for objects.""" 180 | return self.objects() 181 | 182 | 183 | class ZipCodeResponse(Response): 184 | """Represents the data returned from an Analytics API zip endpoint.""" 185 | 186 | def objects(self): 187 | """Gets a list of ZipCode objects for the requested zipcodes, 188 | each containing the zipcodes's returned json data from the API. 189 | 190 | Returns: 191 | List of ZipCode objects 192 | """ 193 | return self._get_objects(ZipCode) 194 | 195 | def zipcodes(self): 196 | """Alias method for objects.""" 197 | return self.objects() 198 | 199 | 200 | class MsaResponse(Response): 201 | """Represents the data returned from an Analytics API msa endpoint.""" 202 | 203 | def objects(self): 204 | """Gets a list of Msa objects for the requested msas, 205 | each containing the msa's returned json data from the API. 206 | 207 | Returns: 208 | List of Msa objects 209 | """ 210 | return self._get_objects(Msa) 211 | 212 | def msas(self): 213 | """Alias method for objects.""" 214 | return self.objects() 215 | 216 | 217 | class ValueReportResponse(Response): 218 | """The response from a value_report request.""" 219 | 220 | def objects(self): 221 | """The value_report endpoint returns a json dict 222 | instead of a list of address results.""" 223 | return [] 224 | 225 | 226 | class RentalReportResponse(Response): 227 | """The response from a rental_report request.""" 228 | 229 | def objects(self): 230 | """The rental_report endpoint returns a json dict 231 | instead of a list of address results.""" 232 | return [] 233 | -------------------------------------------------------------------------------- /housecanary/utilities.py: -------------------------------------------------------------------------------- 1 | """Utility functions for hc-api-python""" 2 | 3 | from datetime import datetime 4 | 5 | 6 | def get_readable_time_string(seconds): 7 | """Returns human readable string from number of seconds""" 8 | seconds = int(seconds) 9 | minutes = seconds // 60 10 | seconds = seconds % 60 11 | hours = minutes // 60 12 | minutes = minutes % 60 13 | days = hours // 24 14 | hours = hours % 24 15 | 16 | result = "" 17 | if days > 0: 18 | result += "%d %s " % (days, "Day" if (days == 1) else "Days") 19 | if hours > 0: 20 | result += "%d %s " % (hours, "Hour" if (hours == 1) else "Hours") 21 | if minutes > 0: 22 | result += "%d %s " % (minutes, "Minute" if (minutes == 1) else "Minutes") 23 | if seconds > 0: 24 | result += "%d %s " % (seconds, "Second" if (seconds == 1) else "Seconds") 25 | 26 | return result.strip() 27 | 28 | 29 | def get_datetime_from_timestamp(timestamp): 30 | """Return datetime from unix timestamp""" 31 | try: 32 | return datetime.fromtimestamp(int(timestamp)) 33 | except: 34 | return None 35 | 36 | 37 | def get_rate_limits(response): 38 | """Returns a list of rate limit information from a given response's headers.""" 39 | periods = response.headers['X-RateLimit-Period'] 40 | if not periods: 41 | return [] 42 | 43 | rate_limits = [] 44 | 45 | periods = periods.split(',') 46 | limits = response.headers['X-RateLimit-Limit'].split(',') 47 | remaining = response.headers['X-RateLimit-Remaining'].split(',') 48 | reset = response.headers['X-RateLimit-Reset'].split(',') 49 | 50 | for idx, period in enumerate(periods): 51 | rate_limit = {} 52 | limit_period = get_readable_time_string(period) 53 | rate_limit["period"] = limit_period 54 | rate_limit["period_seconds"] = period 55 | rate_limit["request_limit"] = limits[idx] 56 | rate_limit["requests_remaining"] = remaining[idx] 57 | 58 | reset_datetime = get_datetime_from_timestamp(reset[idx]) 59 | rate_limit["reset"] = reset_datetime 60 | 61 | right_now = datetime.now() 62 | if (reset_datetime is not None) and (right_now < reset_datetime): 63 | # add 1 second because of rounding 64 | seconds_remaining = (reset_datetime - right_now).seconds + 1 65 | else: 66 | seconds_remaining = 0 67 | 68 | rate_limit["reset_in_seconds"] = seconds_remaining 69 | 70 | rate_limit["time_to_reset"] = get_readable_time_string(seconds_remaining) 71 | rate_limits.append(rate_limit) 72 | 73 | return rate_limits 74 | -------------------------------------------------------------------------------- /notebooks/using-test-credentials.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# HouseCanary API - Using Test API Credentials" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "#### This tutorial will show you how to use the HouseCanary API with test API keys and addresses. First we'll make a GET request using our test API key and secret to retrieve test addresses. Then we'll use those addresses to call the various HouseCanary API endpoints such as the Analytics API, the Value Report API and the Rental Report API." 15 | ] 16 | }, 17 | { 18 | "cell_type": "markdown", 19 | "metadata": {}, 20 | "source": [ 21 | "First, grab your test API key and secret. If you don't already have a test API key and secret, generate them on your [API Settings page](https://valuereport.housecanary.com/settings/api-settings). " 22 | ] 23 | }, 24 | { 25 | "cell_type": "markdown", 26 | "metadata": {}, 27 | "source": [ 28 | "Next, we'll make a GET request to the `test_addresses` endpoint to retrieve a list of test addresses we can use with our test credentials. Keep in the mind, these test addresses can change periodically, so be sure to retrieve the latest ones before trying to use them." 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": 1, 34 | "metadata": { 35 | "collapsed": false 36 | }, 37 | "outputs": [ 38 | { 39 | "name": "stdout", 40 | "output_type": "stream", 41 | "text": [ 42 | "Enter your API secret: ········\n", 43 | "[{u'address': u'7904 Verde Springs Dr', u'zipcode': u'89128'},\n", 44 | " {u'address': u'8691 Flowersong Cv', u'zipcode': u'33473'},\n", 45 | " {u'address': u'22905 Cielo Vis', u'zipcode': u'78255'},\n", 46 | " {u'address': u'16 Thomas Ave', u'zipcode': u'03076'},\n", 47 | " {u'address': u'1111 Oronoco St Unit 441', u'zipcode': u'22314'},\n", 48 | " {u'address': u'2718 16th Ave S', u'zipcode': u'55407'},\n", 49 | " {u'address': u'590 Foothill Rd', u'zipcode': u'95023'},\n", 50 | " {u'address': u'30737 County Road 356-6', u'zipcode': u'81211'},\n", 51 | " {u'address': u'333 N Canal St Apt 2901', u'zipcode': u'60606'},\n", 52 | " {u'address': u'3466 Erie Shore Dr', u'zipcode': u'48162'}]\n" 53 | ] 54 | } 55 | ], 56 | "source": [ 57 | "import requests\n", 58 | "import pprint\n", 59 | "import getpass\n", 60 | "\n", 61 | "test_key = 'test_SX599E3XL0N8P1D30512'\n", 62 | "test_secret = getpass.getpass(prompt='Enter your API secret: ')\n", 63 | "\n", 64 | "url = 'https://api.housecanary.com/v2/property/test_addresses'\n", 65 | "\n", 66 | "response = requests.get(url, auth=(test_key, test_secret))\n", 67 | "\n", 68 | "test_addresses = response.json()\n", 69 | "\n", 70 | "pprint.pprint(test_addresses)\n", 71 | "\n", 72 | "# NOTE: The test addresses change periodically,\n", 73 | "# so the ones shown below may no longer be valid.\n", 74 | "# Call this endpoint yourself to retrieve valid test addresses." 75 | ] 76 | }, 77 | { 78 | "cell_type": "code", 79 | "execution_count": 2, 80 | "metadata": { 81 | "collapsed": false 82 | }, 83 | "outputs": [ 84 | { 85 | "name": "stdout", 86 | "output_type": "stream", 87 | "text": [ 88 | "https://api.housecanary.com/v2/property/details?zipcode=89128&address=7904+Verde+Springs+Dr\n" 89 | ] 90 | } 91 | ], 92 | "source": [ 93 | "# Now that we've retrieved some test addresses, we can use one of them\n", 94 | "# to test out the HouseCanary API endpoints.\n", 95 | "# Let's try one of the Analytics API endpoints.\n", 96 | "\n", 97 | "# we'll just take the first address from the list.\n", 98 | "sample_address = test_addresses[0]\n", 99 | "\n", 100 | "url = 'https://api.housecanary.com/v2/property/details'\n", 101 | "\n", 102 | "params = {'address': sample_address['address'],\n", 103 | " 'zipcode': sample_address['zipcode']}\n", 104 | "\n", 105 | "response = requests.get(url, params=params, auth=(test_key, test_secret))\n", 106 | "\n", 107 | "# We can see what the url looks like\n", 108 | "print(response.url)\n" 109 | ] 110 | }, 111 | { 112 | "cell_type": "code", 113 | "execution_count": 3, 114 | "metadata": { 115 | "collapsed": false 116 | }, 117 | "outputs": [ 118 | { 119 | "name": "stdout", 120 | "output_type": "stream", 121 | "text": [ 122 | "[{u'address_info': {u'address': u'7904 Verde Springs Dr',\n", 123 | " u'address_full': u'7904 Verde Springs Dr Las Vegas NV 89128',\n", 124 | " u'city': u'Las Vegas',\n", 125 | " u'county_fips': u'32003',\n", 126 | " u'geo_precision': u'rooftop',\n", 127 | " u'lat': 36.19194,\n", 128 | " u'lng': -115.26735,\n", 129 | " u'state': u'NV',\n", 130 | " u'unit': None,\n", 131 | " u'zipcode': u'89128',\n", 132 | " u'zipcode_plus4': u'7333'},\n", 133 | " u'property/details': {u'api_code': 0,\n", 134 | " u'api_code_description': u'ok',\n", 135 | " u'result': {u'assessment': {u'assessment_year': 2016,\n", 136 | " u'tax_amount': 1270.51,\n", 137 | " u'tax_year': 2015,\n", 138 | " u'total_assessed_value': 45636.0},\n", 139 | " u'property': {u'air_conditioning': u'central',\n", 140 | " u'attic': None,\n", 141 | " u'basement': None,\n", 142 | " u'building_area_sq_ft': 1190,\n", 143 | " u'exterior_walls': u'stucco',\n", 144 | " u'fireplace': True,\n", 145 | " u'full_bath_count': None,\n", 146 | " u'garage_parking_of_cars': 2,\n", 147 | " u'garage_type_parking': u'attached_garage',\n", 148 | " u'heating': u'forced_air_unit',\n", 149 | " u'heating_fuel_type': None,\n", 150 | " u'no_of_buildings': 0,\n", 151 | " u'no_of_stories': None,\n", 152 | " u'number_of_bedrooms': 3,\n", 153 | " u'number_of_units': 0,\n", 154 | " u'partial_bath_count': None,\n", 155 | " u'pool': None,\n", 156 | " u'property_type': u'Single Family Residential',\n", 157 | " u'sewer': None,\n", 158 | " u'site_area_acres': 0.09,\n", 159 | " u'style': None,\n", 160 | " u'subdivision': u'SANTA FE-PHASE 1',\n", 161 | " u'total_bath_count': 2.0,\n", 162 | " u'total_number_of_rooms': 5,\n", 163 | " u'water': None,\n", 164 | " u'year_built': 1993}}}}]\n" 165 | ] 166 | } 167 | ], 168 | "source": [ 169 | "# print the output\n", 170 | "pprint.pprint(response.json())" 171 | ] 172 | }, 173 | { 174 | "cell_type": "code", 175 | "execution_count": 4, 176 | "metadata": { 177 | "collapsed": false 178 | }, 179 | "outputs": [ 180 | { 181 | "name": "stdout", 182 | "output_type": "stream", 183 | "text": [ 184 | "https://api.housecanary.com/v2/property/value_report?format=json&zipcode=89128&address=7904+Verde+Springs+Dr\n" 185 | ] 186 | } 187 | ], 188 | "source": [ 189 | "# Let's try the Value Report API.\n", 190 | "\n", 191 | "url = 'https://api.housecanary.com/v2/property/value_report'\n", 192 | "\n", 193 | "params = {'address': sample_address['address'],\n", 194 | " 'zipcode': sample_address['zipcode'],\n", 195 | " 'format': 'json'}\n", 196 | "\n", 197 | "response = requests.get(url, params=params, auth=(test_key, test_secret))\n", 198 | "\n", 199 | "print(response.url)\n", 200 | "\n", 201 | "# use response.json() to get the Value Report json. The content is too long to display here." 202 | ] 203 | }, 204 | { 205 | "cell_type": "code", 206 | "execution_count": 5, 207 | "metadata": { 208 | "collapsed": false 209 | }, 210 | "outputs": [ 211 | { 212 | "name": "stdout", 213 | "output_type": "stream", 214 | "text": [ 215 | "https://api.housecanary.com/v2/property/rental_report?format=json&zipcode=89128&address=7904+Verde+Springs+Dr\n" 216 | ] 217 | } 218 | ], 219 | "source": [ 220 | "# Finally, let's try the Rental Report API.\n", 221 | "\n", 222 | "url = 'https://api.housecanary.com/v2/property/rental_report'\n", 223 | "\n", 224 | "params = {'address': sample_address['address'],\n", 225 | " 'zipcode': sample_address['zipcode'],\n", 226 | " 'format': 'json'}\n", 227 | "\n", 228 | "response = requests.get(url, params=params, auth=(test_key, test_secret))\n", 229 | "\n", 230 | "print(response.url)\n", 231 | "\n", 232 | "# use response.json() to get the Rental Report json. The content is too long to display here." 233 | ] 234 | } 235 | ], 236 | "metadata": { 237 | "kernelspec": { 238 | "display_name": "Python 2", 239 | "language": "python", 240 | "name": "python2" 241 | }, 242 | "language_info": { 243 | "codemirror_mode": { 244 | "name": "ipython", 245 | "version": 2 246 | }, 247 | "file_extension": ".py", 248 | "mimetype": "text/x-python", 249 | "name": "python", 250 | "nbconvert_exporter": "python", 251 | "pygments_lexer": "ipython2", 252 | "version": "2.7.9" 253 | } 254 | }, 255 | "nbformat": 4, 256 | "nbformat_minor": 0 257 | } 258 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | requests 2 | docopt==0.6.2 3 | openpyxl==2.4.0 4 | python-slugify 5 | -------------------------------------------------------------------------------- /sample_input/sample-input-blocks.csv: -------------------------------------------------------------------------------- 1 | block_id,property_type 2 | 060376703241005,SFD 3 | 060374632002006,SFD -------------------------------------------------------------------------------- /sample_input/sample-input-city-state.csv: -------------------------------------------------------------------------------- 1 | address,city,state 2 | 43 Valmonte Plaza,Palos Verdes Estates,CA 3 | 244 S ALTADENA DR,Pasadena,CA -------------------------------------------------------------------------------- /sample_input/sample-input-msas.csv: -------------------------------------------------------------------------------- 1 | msa 2 | 31080 3 | 29820 -------------------------------------------------------------------------------- /sample_input/sample-input-slugs.csv: -------------------------------------------------------------------------------- 1 | slug,client_value,client_value_sqft 2 | 43-Valmonte-Plz-Palos-Verdes-Estates-CA-90274,1000000,2000 3 | 244-S-Altadena-Dr-Pasadena-CA-91107,1200000,2500 -------------------------------------------------------------------------------- /sample_input/sample-input-zipcodes.csv: -------------------------------------------------------------------------------- 1 | zipcode,meta 2 | 90274,"Area 1" 3 | 91107,"Area 2" -------------------------------------------------------------------------------- /sample_input/sample-input.csv: -------------------------------------------------------------------------------- 1 | address,zipcode 2 | 43 Valmonte Plaza,90274 3 | 244 S ALTADENA DR,91107 4 | 399 HAMILTON AVE,91106 5 | 3510 THORNDALE RD,91107 6 | 413 PLEASANT HILL LN,91024 7 | 2034 VISTA AVE,91006 8 | 2024 THOREAU ST,90047 9 | 136 SAN MIGUEL DR,91007 10 | 420 VAQUERO RD,91007 11 | 303 LELAND WAY,91006 12 | 896 S GOLDEN WEST AVE,91007 13 | 482 W LE ROY AVE,91007 14 | 2428 CROSS ST,91214 15 | 2348 CHAPMAN RD,91214 16 | 2512 MARY ST,91020 17 | 2055 RANCHO CANADA PL,91011 18 | 4349 COBBLESTONE LN,91011 -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | import io 2 | import os 3 | from platform import python_version 4 | import re 5 | 6 | from setuptools import find_packages, setup 7 | 8 | 9 | def read(*parts): 10 | return io.open(os.path.join(*parts), 'rb').read().decode('UTF-8') 11 | 12 | 13 | def find_version(*file_paths): 14 | version_file = read(*file_paths) 15 | version_match = re.search(r"^__version__ = ['\"]([^'\"]*)['\"]", 16 | version_file, re.M) 17 | if version_match: 18 | return version_match.group(1) 19 | raise RuntimeError("Unable to find version string.") 20 | 21 | 22 | def test_requirements(): 23 | reqs = ['nose', 'requests-mock'] 24 | if python_version().startswith('2'): 25 | reqs.append('mock') 26 | 27 | return reqs 28 | 29 | 30 | setup(name='housecanary', 31 | version=find_version('housecanary', '__init__.py'), 32 | description='Client Wrapper for the HouseCanary API', 33 | long_description=read('README.rst'), 34 | url='http://github.com/housecanary/hc-api-python', 35 | author='HouseCanary', 36 | author_email='techops@housecanary.com', 37 | license='MIT', 38 | packages=find_packages(include=['housecanary', 'housecanary.*']), 39 | install_requires=['requests', 'docopt', 'openpyxl', 'python-slugify', 'future'], 40 | zip_safe=False, 41 | test_suite='nose.collector', 42 | tests_require=test_requirements(), 43 | entry_points={ 44 | 'console_scripts': [ 45 | 'hc_api_excel_concat=housecanary.hc_api_excel_concat.hc_api_excel_concat:main', 46 | 'hc_api_export=housecanary.hc_api_export.hc_api_export:main' 47 | ] 48 | }, 49 | classifiers=[ 50 | 'Programming Language :: Python :: 2.7', 51 | 'Programming Language :: Python :: 3.5', 52 | 'Programming Language :: Python :: 3.6', 53 | ],) 54 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/housecanary/hc-api-python/2bb9e2208b34e8617575de45934357ee33b8531c/tests/__init__.py -------------------------------------------------------------------------------- /tests/test_excel_utilities.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | from housecanary.excel import utilities 3 | 4 | 5 | class ExcelUtilitiesTestCase(unittest.TestCase): 6 | """Tests for the Excel Utilities""" 7 | 8 | def test_normalize_cell_value_for_dict(self): 9 | result = utilities.normalize_cell_value({'test': 'value'}) 10 | self.assertEqual('{"test": "value"}', result) 11 | 12 | def test_normalize_cell_value_for_list(self): 13 | result = utilities.normalize_cell_value([{'test1': 'value1'}, {'test2': 'value2'}]) 14 | self.assertEqual('[{"test1": "value1"}, {"test2": "value2"}]', result) 15 | 16 | def test_normalize_cell_value_for_other_type(self): 17 | result = utilities.normalize_cell_value(5) 18 | self.assertEqual(5, result) 19 | 20 | def test_convert_snake_to_title_case(self): 21 | result = utilities.convert_snake_to_title_case('address_full_value') 22 | self.assertEqual('Address Full Value', result) 23 | 24 | def test_convert_title_to_snake_case(self): 25 | result = utilities.convert_title_to_snake_case('Address Full Value') 26 | self.assertEqual('address_full_value', result) 27 | 28 | def test_get_addresses_from_input_file(self): 29 | result = utilities.get_addresses_from_input_file('./tests/test_files/test_input.csv') 30 | self.assertEqual(('43 Valmonte Plaza', '90274'), result[0]) 31 | self.assertEqual(('244 S ALTADENA DR', '91107'), result[1]) 32 | 33 | def test_get_identifiers_from_input_file_with_extra_field(self): 34 | result = utilities.get_identifiers_from_input_file('./tests/test_files/test_input.csv') 35 | self.assertEqual({'zipcode': '90274', 'address': '43 Valmonte Plaza'}, result[0]) 36 | self.assertEqual({'zipcode': '91107', 'address': '244 S ALTADENA DR'}, result[1]) 37 | 38 | def test_get_identifiers_from_input_file_with_block_ids(self): 39 | result = utilities.get_identifiers_from_input_file('./sample_input/sample-input-blocks.csv') 40 | self.assertEqual({'property_type': 'SFD', 'block_id': '060376703241005'}, result[0]) 41 | self.assertEqual({'property_type': 'SFD', 'block_id': '060374632002006'}, result[1]) 42 | 43 | def test_get_identifiers_from_input_file_with_city_state(self): 44 | result = utilities.get_identifiers_from_input_file( 45 | './sample_input/sample-input-city-state.csv' 46 | ) 47 | self.assertEqual( 48 | {'address': '43 Valmonte Plaza', 'city': 'Palos Verdes Estates', 'state': 'CA'}, 49 | result[0] 50 | ) 51 | self.assertEqual( 52 | {'address': '244 S ALTADENA DR', 'city': 'Pasadena', 'state': 'CA'}, 53 | result[1] 54 | ) 55 | 56 | def test_get_identifiers_from_input_file_with_msas(self): 57 | result = utilities.get_identifiers_from_input_file('./sample_input/sample-input-msas.csv') 58 | self.assertEqual({'msa': '31080'}, result[0]) 59 | self.assertEqual({'msa': '29820'}, result[1]) 60 | 61 | def test_get_identifiers_from_input_file_with_slugs(self): 62 | result = utilities.get_identifiers_from_input_file('./sample_input/sample-input-slugs.csv') 63 | self.assertEqual( 64 | {'client_value': '1000000', 65 | 'client_value_sqft': '2000', 66 | 'slug': '43-Valmonte-Plz-Palos-Verdes-Estates-CA-90274'}, 67 | result[0] 68 | ) 69 | self.assertEqual( 70 | {'client_value': '1200000', 71 | 'client_value_sqft': '2500', 72 | 'slug': '244-S-Altadena-Dr-Pasadena-CA-91107'}, 73 | result[1] 74 | ) 75 | 76 | def test_get_identifiers_from_input_file_with_zipcodes(self): 77 | result = utilities.get_identifiers_from_input_file('./sample_input/sample-input-zipcodes.csv') 78 | self.assertEqual({'meta': 'Area 1', 'zipcode': '90274'}, result[0]) 79 | self.assertEqual({'meta': 'Area 2', 'zipcode': '91107'}, result[1]) 80 | -------------------------------------------------------------------------------- /tests/test_files/test_excel.xlsx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/housecanary/hc-api-python/2bb9e2208b34e8617575de45934357ee33b8531c/tests/test_files/test_excel.xlsx -------------------------------------------------------------------------------- /tests/test_files/test_input.csv: -------------------------------------------------------------------------------- 1 | address,zipcode,other_field 2 | 43 Valmonte Plaza,90274,field1 3 | 244 S ALTADENA DR,91107,field2 -------------------------------------------------------------------------------- /tests/test_hc_api_excel_concat.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | import requests_mock 3 | import os 4 | import io 5 | import shutil 6 | from housecanary.hc_api_excel_concat import hc_api_excel_concat 7 | 8 | 9 | class HcApiExcelConcatTestCase(unittest.TestCase): 10 | """Tests for the hc_api_excel_concat tool""" 11 | 12 | def setUp(self): 13 | self.output_file = 'unittest_hc_api_excel_concat.xlsx' 14 | self.output_path = 'unittest_housecanary_excel' 15 | self._remove_test_output() 16 | self.headers = {"content-type": "application/xlsx"} 17 | 18 | def tearDown(self): 19 | self._remove_test_output() 20 | 21 | def _remove_test_output(self): 22 | if os.path.exists(self.output_file): 23 | os.remove(self.output_file) 24 | if os.path.exists(self.output_path): 25 | shutil.rmtree(self.output_path) 26 | 27 | def get_docopt_args(self, endpoint): 28 | return { 29 | '--endpoint': endpoint, 30 | '--files': self.output_path, 31 | '--help': False, 32 | '--key': None, 33 | '--output': self.output_file, 34 | '--retry': False, 35 | '--secret': None, 36 | '--type': None, 37 | '-h': False, 38 | '': './tests/test_files/test_input.csv' 39 | } 40 | 41 | def test_hc_api_excel_concat_value_report(self): 42 | with io.open('tests/test_files/test_excel.xlsx', 'rb') as f: 43 | content = f.read() 44 | with requests_mock.Mocker() as m: 45 | m.get("/v2/property/value_report", headers=self.headers, content=content) 46 | self.assertFalse(os.path.exists(os.path.join(self.output_path, self.output_file))) 47 | self.assertFalse(os.path.exists(self.output_path)) 48 | hc_api_excel_concat.hc_api_excel_concat(self.get_docopt_args('value_report')) 49 | self.assertTrue(os.path.exists(os.path.join(self.output_path, self.output_file))) 50 | self.assertTrue(os.path.exists(self.output_path)) 51 | 52 | def test_hc_api_excel_concat_rental_report(self): 53 | with io.open('tests/test_files/test_excel.xlsx', 'rb') as f: 54 | content = f.read() 55 | with requests_mock.Mocker() as m: 56 | m.get("/v2/property/rental_report", headers=self.headers, content=content) 57 | self.assertFalse(os.path.exists(os.path.join(self.output_path, self.output_file))) 58 | self.assertFalse(os.path.exists(self.output_path)) 59 | hc_api_excel_concat.hc_api_excel_concat(self.get_docopt_args('rental_report')) 60 | self.assertTrue(os.path.exists(os.path.join(self.output_path, self.output_file))) 61 | self.assertTrue(os.path.exists(self.output_path)) 62 | -------------------------------------------------------------------------------- /tests/test_hc_api_export.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | import requests_mock 3 | import os 4 | import shutil 5 | from housecanary.hc_api_export import hc_api_export 6 | 7 | 8 | @requests_mock.Mocker() 9 | class HcApiExportTestCase(unittest.TestCase): 10 | """Tests for the hc_api_export tool""" 11 | 12 | def setUp(self): 13 | self.output_excel_file = 'unittest_hc_api_export.xlsx' 14 | self.output_csv_path = 'unittest_housecanary_csv' 15 | self._remove_test_output() 16 | self.headers = {"content-type": "application/json"} 17 | # self.response = [{'property/value': {'api_code_description': 'ok', 'result': {'value': 'test'}}, 'address_info': 'test'}] 18 | 19 | def tearDown(self): 20 | self._remove_test_output() 21 | 22 | def _remove_test_output(self): 23 | if os.path.exists(self.output_excel_file): 24 | os.remove(self.output_excel_file) 25 | if os.path.exists(self.output_csv_path): 26 | shutil.rmtree(self.output_csv_path) 27 | 28 | def get_docopt_args(self, output_type, endpoints, input_file): 29 | return { 30 | '--help': False, 31 | '--key': None, 32 | '--output': self.output_excel_file, 33 | '--path': self.output_csv_path, 34 | '--retry': False, 35 | '--secret': None, 36 | '--type': output_type, 37 | '-h': False, 38 | '': endpoints, 39 | '': input_file 40 | } 41 | 42 | def _test_excel_output(self, docopt_args): 43 | self.assertFalse(os.path.exists(self.output_excel_file)) 44 | hc_api_export.hc_api_export(docopt_args) 45 | self.assertTrue(os.path.exists(self.output_excel_file)) 46 | 47 | def _test_csv_output(self, docopt_args): 48 | self.assertFalse(os.path.exists(self.output_csv_path)) 49 | hc_api_export.hc_api_export(docopt_args) 50 | self.assertTrue(os.path.exists(self.output_csv_path)) 51 | 52 | def test_hc_api_export_property_excel(self, mock): 53 | response = [{'address_info': {'city': 'Palos Verdes Estates', 'county_fips': '06037', 'geo_precision': 'rooftop', 'block_id': '060376703241005', 'zipcode': '90274', 'address_full': '43 Valmonte Plz Palos Verdes Estates CA 90274', 'zipcode_plus4': '1444', 'state': 'CA', 'metrodiv': '31084', 'unit': None, 'address': '43 Valmonte Plz', 'lat': 33.79814, 'lng': -118.36455, 'slug': '43-Valmonte-Plz-Palos-Verdes-Estates-CA-90274', 'msa': '31080'}, 'property/value': {'api_code_description': 'ok', 'api_code': 0, 'result': {'value': {'price_upr': 1780318, 'price_lwr': 1505350, 'price_mean': 1642834, 'fsd': 0.0836871}}}}, {'address_info': {'city': 'Pasadena', 'county_fips': '06037', 'geo_precision': 'rooftop', 'block_id': '060374632002006', 'zipcode': '91107', 'address_full': '244 S Altadena Dr Pasadena CA 91107', 'zipcode_plus4': '4820', 'state': 'CA', 'metrodiv': '31084', 'unit': None, 'address': '244 S Altadena Dr', 'lat': 34.14182, 'lng': -118.09833, 'slug': '244-S-Altadena-Dr-Pasadena-CA-91107', 'msa': '31080'}, 'property/value': {'api_code_description': 'ok', 'api_code': 0, 'result': {'value': {'price_upr': 672556, 'price_lwr': 615548, 'price_mean': 644052, 'fsd': 0.0442573}}}}] 54 | mock.post("/v2/property/component_mget", headers=self.headers, json=response) 55 | self._test_excel_output(self.get_docopt_args( 56 | 'excel', 'property/*', './tests/test_files/test_input.csv' 57 | )) 58 | 59 | def test_hc_api_export_property_csv(self, mock): 60 | response = [{'address_info': {'city': 'Palos Verdes Estates', 'county_fips': '06037', 'geo_precision': 'rooftop', 'block_id': '060376703241005', 'zipcode': '90274', 'address_full': '43 Valmonte Plz Palos Verdes Estates CA 90274', 'zipcode_plus4': '1444', 'state': 'CA', 'metrodiv': '31084', 'unit': None, 'address': '43 Valmonte Plz', 'lat': 33.79814, 'lng': -118.36455, 'slug': '43-Valmonte-Plz-Palos-Verdes-Estates-CA-90274', 'msa': '31080'}, 'property/value': {'api_code_description': 'ok', 'api_code': 0, 'result': {'value': {'price_upr': 1780318, 'price_lwr': 1505350, 'price_mean': 1642834, 'fsd': 0.0836871}}}}, {'address_info': {'city': 'Pasadena', 'county_fips': '06037', 'geo_precision': 'rooftop', 'block_id': '060374632002006', 'zipcode': '91107', 'address_full': '244 S Altadena Dr Pasadena CA 91107', 'zipcode_plus4': '4820', 'state': 'CA', 'metrodiv': '31084', 'unit': None, 'address': '244 S Altadena Dr', 'lat': 34.14182, 'lng': -118.09833, 'slug': '244-S-Altadena-Dr-Pasadena-CA-91107', 'msa': '31080'}, 'property/value': {'api_code_description': 'ok', 'api_code': 0, 'result': {'value': {'price_upr': 672556, 'price_lwr': 615548, 'price_mean': 644052, 'fsd': 0.0442573}}}}] 61 | mock.post("/v2/property/component_mget", headers=self.headers, json=response) 62 | self._test_csv_output(self.get_docopt_args( 63 | 'csv', 'property/*', './tests/test_files/test_input.csv' 64 | )) 65 | 66 | def test_hc_api_export_excel_property_city_state(self, mock): 67 | response = [{'address_info': {'city': 'Palos Verdes Estates', 'county_fips': '06037', 'geo_precision': 'rooftop', 'block_id': '060376703241005', 'zipcode': '90274', 'address_full': '43 Valmonte Plz Palos Verdes Estates CA 90274', 'zipcode_plus4': '1444', 'state': 'CA', 'metrodiv': '31084', 'unit': None, 'address': '43 Valmonte Plz', 'lat': 33.79814, 'lng': -118.36455, 'slug': '43-Valmonte-Plz-Palos-Verdes-Estates-CA-90274', 'msa': '31080'}, 'property/value': {'api_code_description': 'ok', 'api_code': 0, 'result': {'value': {'price_upr': 1780318, 'price_lwr': 1505350, 'price_mean': 1642834, 'fsd': 0.0836871}}}}, {'address_info': {'city': 'Pasadena', 'county_fips': '06037', 'geo_precision': 'rooftop', 'block_id': '060374632002006', 'zipcode': '91107', 'address_full': '244 S Altadena Dr Pasadena CA 91107', 'zipcode_plus4': '4820', 'state': 'CA', 'metrodiv': '31084', 'unit': None, 'address': '244 S Altadena Dr', 'lat': 34.14182, 'lng': -118.09833, 'slug': '244-S-Altadena-Dr-Pasadena-CA-91107', 'msa': '31080'}, 'property/value': {'api_code_description': 'ok', 'api_code': 0, 'result': {'value': {'price_upr': 672556, 'price_lwr': 615548, 'price_mean': 644052, 'fsd': 0.0442573}}}}] 68 | mock.post("/v2/property/component_mget", headers=self.headers, json=response) 69 | self._test_excel_output(self.get_docopt_args( 70 | 'excel', 'property/*', './sample_input/sample-input-city-state.csv' 71 | )) 72 | 73 | def test_hc_api_export_csv_property_city_state(self, mock): 74 | response = [{'address_info': {'city': 'Palos Verdes Estates', 'county_fips': '06037', 'geo_precision': 'rooftop', 'block_id': '060376703241005', 'zipcode': '90274', 'address_full': '43 Valmonte Plz Palos Verdes Estates CA 90274', 'zipcode_plus4': '1444', 'state': 'CA', 'metrodiv': '31084', 'unit': None, 'address': '43 Valmonte Plz', 'lat': 33.79814, 'lng': -118.36455, 'slug': '43-Valmonte-Plz-Palos-Verdes-Estates-CA-90274', 'msa': '31080'}, 'property/value': {'api_code_description': 'ok', 'api_code': 0, 'result': {'value': {'price_upr': 1780318, 'price_lwr': 1505350, 'price_mean': 1642834, 'fsd': 0.0836871}}}}, {'address_info': {'city': 'Pasadena', 'county_fips': '06037', 'geo_precision': 'rooftop', 'block_id': '060374632002006', 'zipcode': '91107', 'address_full': '244 S Altadena Dr Pasadena CA 91107', 'zipcode_plus4': '4820', 'state': 'CA', 'metrodiv': '31084', 'unit': None, 'address': '244 S Altadena Dr', 'lat': 34.14182, 'lng': -118.09833, 'slug': '244-S-Altadena-Dr-Pasadena-CA-91107', 'msa': '31080'}, 'property/value': {'api_code_description': 'ok', 'api_code': 0, 'result': {'value': {'price_upr': 672556, 'price_lwr': 615548, 'price_mean': 644052, 'fsd': 0.0442573}}}}] 75 | mock.post("/v2/property/component_mget", headers=self.headers, json=response) 76 | self._test_csv_output(self.get_docopt_args( 77 | 'csv', 'property/*', './sample_input/sample-input-city-state.csv' 78 | )) 79 | 80 | def test_hc_api_export_excel_property_slug(self, mock): 81 | response = [{'address_info': {'city': 'Palos Verdes Estates', 'county_fips': '06037', 'geo_precision': 'rooftop', 'block_id': '060376703241005', 'zipcode': '90274', 'address_full': '43 Valmonte Plz Palos Verdes Estates CA 90274', 'zipcode_plus4': '1444', 'state': 'CA', 'metrodiv': '31084', 'unit': None, 'address': '43 Valmonte Plz', 'lat': 33.79814, 'lng': -118.36455, 'slug': '43-Valmonte-Plz-Palos-Verdes-Estates-CA-90274', 'msa': '31080'}, 'property/value': {'api_code_description': 'ok', 'api_code': 0, 'result': {'value': {'price_upr': 1780318, 'price_lwr': 1505350, 'price_mean': 1642834, 'fsd': 0.0836871}}}}, {'address_info': {'city': 'Pasadena', 'county_fips': '06037', 'geo_precision': 'rooftop', 'block_id': '060374632002006', 'zipcode': '91107', 'address_full': '244 S Altadena Dr Pasadena CA 91107', 'zipcode_plus4': '4820', 'state': 'CA', 'metrodiv': '31084', 'unit': None, 'address': '244 S Altadena Dr', 'lat': 34.14182, 'lng': -118.09833, 'slug': '244-S-Altadena-Dr-Pasadena-CA-91107', 'msa': '31080'}, 'property/value': {'api_code_description': 'ok', 'api_code': 0, 'result': {'value': {'price_upr': 672556, 'price_lwr': 615548, 'price_mean': 644052, 'fsd': 0.0442573}}}}] 82 | mock.post("/v2/property/component_mget", headers=self.headers, json=response) 83 | self._test_excel_output(self.get_docopt_args( 84 | 'excel', 'property/*', './sample_input/sample-input-slugs.csv' 85 | )) 86 | 87 | def test_hc_api_export_csv_property_slug(self, mock): 88 | response = [{'address_info': {'city': 'Palos Verdes Estates', 'county_fips': '06037', 'geo_precision': 'rooftop', 'block_id': '060376703241005', 'zipcode': '90274', 'address_full': '43 Valmonte Plz Palos Verdes Estates CA 90274', 'zipcode_plus4': '1444', 'state': 'CA', 'metrodiv': '31084', 'unit': None, 'address': '43 Valmonte Plz', 'lat': 33.79814, 'lng': -118.36455, 'slug': '43-Valmonte-Plz-Palos-Verdes-Estates-CA-90274', 'msa': '31080'}, 'property/value': {'api_code_description': 'ok', 'api_code': 0, 'result': {'value': {'price_upr': 1780318, 'price_lwr': 1505350, 'price_mean': 1642834, 'fsd': 0.0836871}}}}, {'address_info': {'city': 'Pasadena', 'county_fips': '06037', 'geo_precision': 'rooftop', 'block_id': '060374632002006', 'zipcode': '91107', 'address_full': '244 S Altadena Dr Pasadena CA 91107', 'zipcode_plus4': '4820', 'state': 'CA', 'metrodiv': '31084', 'unit': None, 'address': '244 S Altadena Dr', 'lat': 34.14182, 'lng': -118.09833, 'slug': '244-S-Altadena-Dr-Pasadena-CA-91107', 'msa': '31080'}, 'property/value': {'api_code_description': 'ok', 'api_code': 0, 'result': {'value': {'price_upr': 672556, 'price_lwr': 615548, 'price_mean': 644052, 'fsd': 0.0442573}}}}] 89 | mock.post("/v2/property/component_mget", headers=self.headers, json=response) 90 | self._test_csv_output(self.get_docopt_args( 91 | 'csv', 'property/*', './sample_input/sample-input-slugs.csv' 92 | )) 93 | 94 | def test_hc_api_export_zip_excel(self, mock): 95 | response = [{'zip/details': {'api_code_description': 'ok', 'api_code': 0, 'result': {'multi_family': {'inventory_total': None, 'price_median': None, 'estimated_sales_total': None, 'market_action_median': None, 'months_of_inventory_median': None, 'days_on_market_median': None}, 'single_family': {'inventory_total': 78.538, 'price_median': 2748899.085, 'estimated_sales_total': None, 'market_action_median': 62.34, 'months_of_inventory_median': None, 'days_on_market_median': 116.132}}}, 'meta': 'Area 1', 'zipcode_info': {'zipcode': '90274'}}, {'zip/details': {'api_code_description': 'ok', 'api_code': 0, 'result': {'multi_family': {'inventory_total': 11.385, 'price_median': 581361.538, 'estimated_sales_total': 8.46, 'market_action_median': 69.34, 'months_of_inventory_median': 1.346, 'days_on_market_median': 29.615}, 'single_family': {'inventory_total': 54.385, 'price_median': 1148846.077, 'estimated_sales_total': 22.229, 'market_action_median': 57.87, 'months_of_inventory_median': 2.447, 'days_on_market_median': 74.846}}}, 'meta': 'Area 2', 'zipcode_info': {'zipcode': '91107'}}] 96 | mock.post("/v2/zip/component_mget", headers=self.headers, json=response) 97 | self._test_excel_output(self.get_docopt_args( 98 | 'excel', 'zip/*', './sample_input/sample-input-zipcodes.csv' 99 | )) 100 | 101 | def test_hc_api_export_zip_csv(self, mock): 102 | response = [{'zip/details': {'api_code_description': 'ok', 'api_code': 0, 'result': {'multi_family': {'inventory_total': None, 'price_median': None, 'estimated_sales_total': None, 'market_action_median': None, 'months_of_inventory_median': None, 'days_on_market_median': None}, 'single_family': {'inventory_total': 78.538, 'price_median': 2748899.085, 'estimated_sales_total': None, 'market_action_median': 62.34, 'months_of_inventory_median': None, 'days_on_market_median': 116.132}}}, 'meta': 'Area 1', 'zipcode_info': {'zipcode': '90274'}}, {'zip/details': {'api_code_description': 'ok', 'api_code': 0, 'result': {'multi_family': {'inventory_total': 11.385, 'price_median': 581361.538, 'estimated_sales_total': 8.46, 'market_action_median': 69.34, 'months_of_inventory_median': 1.346, 'days_on_market_median': 29.615}, 'single_family': {'inventory_total': 54.385, 'price_median': 1148846.077, 'estimated_sales_total': 22.229, 'market_action_median': 57.87, 'months_of_inventory_median': 2.447, 'days_on_market_median': 74.846}}}, 'meta': 'Area 2', 'zipcode_info': {'zipcode': '91107'}}] 103 | mock.post("/v2/zip/component_mget", headers=self.headers, json=response) 104 | self._test_csv_output(self.get_docopt_args( 105 | 'csv', 'zip/*', './sample_input/sample-input-zipcodes.csv' 106 | )) 107 | 108 | def test_hc_api_export_block_excel(self, mock): 109 | response = [{'block/value_distribution': {'api_code_description': 'ok', 'api_code': 0, 'result': {'value_sqft_50': 814.2, 'value_min': 967440, 'value_25': 1141837, 'value_sqft_max': 900.7, 'value_sqft_mean': 812.4, 'value_sqft_count': 13, 'value_sqft_min': 709.2, 'value_max': 2141477, 'value_sqft_25': 798.0, 'value_sqft_95': 883.4, 'value_5': 1043104, 'value_95': 1883893, 'value_50': 1358392, 'value_count': 13, 'value_sd': 312382, 'value_sqft_5': 724.2, 'value_mean': 1382563, 'value_sqft_75': 842.4, 'property_type': 'SFD', 'value_75': 1445081, 'value_sqft_sd': 51.5}}, 'block_info': {'block_id': '060376703241005'}}, {'block/value_distribution': {'api_code_description': 'ok', 'api_code': 0, 'result': {'value_sqft_50': 562.0, 'value_min': 644052, 'value_25': 688884, 'value_sqft_max': 670.5, 'value_sqft_mean': 591.1, 'value_sqft_count': 6, 'value_sqft_min': 542.2, 'value_max': 1214631, 'value_sqft_25': 561.8, 'value_sqft_95': 664.9, 'value_5': 653522, 'value_95': 1102067, 'value_50': 713156, 'value_count': 6, 'value_sd': 212494, 'value_sqft_5': 547.1, 'value_mean': 788550, 'value_sqft_75': 626.5, 'property_type': 'SFD', 'value_75': 752425, 'value_sqft_sd': 53.8}}, 'block_info': {'block_id': '060374632002006'}}] 110 | mock.post("/v2/block/component_mget", headers=self.headers, json=response) 111 | self._test_excel_output(self.get_docopt_args( 112 | 'excel', 'block/*', './sample_input/sample-input-blocks.csv' 113 | )) 114 | 115 | def test_hc_api_export_block_csv(self, mock): 116 | response = [{'block/value_distribution': {'api_code_description': 'ok', 'api_code': 0, 'result': {'value_sqft_50': 814.2, 'value_min': 967440, 'value_25': 1141837, 'value_sqft_max': 900.7, 'value_sqft_mean': 812.4, 'value_sqft_count': 13, 'value_sqft_min': 709.2, 'value_max': 2141477, 'value_sqft_25': 798.0, 'value_sqft_95': 883.4, 'value_5': 1043104, 'value_95': 1883893, 'value_50': 1358392, 'value_count': 13, 'value_sd': 312382, 'value_sqft_5': 724.2, 'value_mean': 1382563, 'value_sqft_75': 842.4, 'property_type': 'SFD', 'value_75': 1445081, 'value_sqft_sd': 51.5}}, 'block_info': {'block_id': '060376703241005'}}, {'block/value_distribution': {'api_code_description': 'ok', 'api_code': 0, 'result': {'value_sqft_50': 562.0, 'value_min': 644052, 'value_25': 688884, 'value_sqft_max': 670.5, 'value_sqft_mean': 591.1, 'value_sqft_count': 6, 'value_sqft_min': 542.2, 'value_max': 1214631, 'value_sqft_25': 561.8, 'value_sqft_95': 664.9, 'value_5': 653522, 'value_95': 1102067, 'value_50': 713156, 'value_count': 6, 'value_sd': 212494, 'value_sqft_5': 547.1, 'value_mean': 788550, 'value_sqft_75': 626.5, 'property_type': 'SFD', 'value_75': 752425, 'value_sqft_sd': 53.8}}, 'block_info': {'block_id': '060374632002006'}}] 117 | mock.post("/v2/block/component_mget", headers=self.headers, json=response) 118 | self._test_csv_output(self.get_docopt_args( 119 | 'csv', 'block/*', './sample_input/sample-input-blocks.csv' 120 | )) 121 | 122 | def test_hc_api_export_msa_excel(self, mock): 123 | response = [{'msa/details': {'api_code_description': 'ok', 'api_code': 0, 'result': {'cagr_1': 0.0697, 'cagr_10': 0.0036, 'cagr_5': 0.0984, 'returns_10': 0.0363, 'max_12mo_loss': -0.217967, 'risk_12mo_loss': 0.088535, 'returns_1': 0.0697, 'cagr_20': 0.0648, 'returns_5': 0.5991}}, 'msa_info': {'msa_name': 'Los Angeles-Long Beach-Anaheim, CA', 'msa': '31080'}}, {'msa/details': {'api_code_description': 'ok', 'api_code': 0, 'result': {'cagr_1': 0.0738, 'cagr_10': -0.0299, 'cagr_5': 0.1406, 'returns_10': -0.2615, 'max_12mo_loss': -0.355033, 'risk_12mo_loss': 0.108988, 'returns_1': 0.0738, 'cagr_20': 0.0276, 'returns_5': 0.9303}}, 'msa_info': {'msa_name': 'Las Vegas-Henderson-Paradise, NV', 'msa': '29820'}}] 124 | mock.post("/v2/msa/component_mget", headers=self.headers, json=response) 125 | self._test_excel_output(self.get_docopt_args( 126 | 'excel', 'msa/*', './sample_input/sample-input-msas.csv' 127 | )) 128 | 129 | def test_hc_api_export_msa_csv(self, mock): 130 | response = [{'msa/details': {'api_code_description': 'ok', 'api_code': 0, 'result': {'cagr_1': 0.0697, 'cagr_10': 0.0036, 'cagr_5': 0.0984, 'returns_10': 0.0363, 'max_12mo_loss': -0.217967, 'risk_12mo_loss': 0.088535, 'returns_1': 0.0697, 'cagr_20': 0.0648, 'returns_5': 0.5991}}, 'msa_info': {'msa_name': 'Los Angeles-Long Beach-Anaheim, CA', 'msa': '31080'}}, {'msa/details': {'api_code_description': 'ok', 'api_code': 0, 'result': {'cagr_1': 0.0738, 'cagr_10': -0.0299, 'cagr_5': 0.1406, 'returns_10': -0.2615, 'max_12mo_loss': -0.355033, 'risk_12mo_loss': 0.108988, 'returns_1': 0.0738, 'cagr_20': 0.0276, 'returns_5': 0.9303}}, 'msa_info': {'msa_name': 'Las Vegas-Henderson-Paradise, NV', 'msa': '29820'}}] 131 | mock.post("/v2/msa/component_mget", headers=self.headers, json=response) 132 | self._test_csv_output(self.get_docopt_args( 133 | 'csv', 'msa/*', './sample_input/sample-input-msas.csv' 134 | )) 135 | 136 | def test_hc_api_export_property_single_endpoint(self, mock): 137 | response = [{'address_info': {'city': 'Palos Verdes Estates', 'county_fips': '06037', 'geo_precision': 'rooftop', 'block_id': '060376703241005', 'zipcode': '90274', 'address_full': '43 Valmonte Plz Palos Verdes Estates CA 90274', 'zipcode_plus4': '1444', 'state': 'CA', 'metrodiv': '31084', 'unit': None, 'address': '43 Valmonte Plz', 'lat': 33.79814, 'lng': -118.36455, 'slug': '43-Valmonte-Plz-Palos-Verdes-Estates-CA-90274', 'msa': '31080'}, 'property/value': {'api_code_description': 'ok', 'api_code': 0, 'result': {'value': {'price_upr': 1780318, 'price_lwr': 1505350, 'price_mean': 1642834, 'fsd': 0.0836871}}}}] 138 | mock.post("/v2/property/value", headers=self.headers, json=response) 139 | self._test_excel_output(self.get_docopt_args( 140 | 'excel', 'property/value', './tests/test_files/test_input.csv' 141 | )) 142 | -------------------------------------------------------------------------------- /tests/test_object.py: -------------------------------------------------------------------------------- 1 | # pylint: disable=missing-docstring 2 | 3 | import unittest 4 | from housecanary.object import Property 5 | from housecanary.object import Block 6 | from housecanary.object import ZipCode 7 | from housecanary.object import Msa 8 | 9 | 10 | class PropertyTestCase(unittest.TestCase): 11 | def setUp(self): 12 | self.test_json = { 13 | 'property/value': { 14 | 'api_code_description': 'ok', 15 | 'api_code': 0, 16 | 'result': { 17 | 'price_pr': 2938.0, 'price_lwr': 2160.0, 'price_mean': 2296.0, 'fsd': 0.17 18 | } 19 | }, 20 | 'address_info': { 21 | 'city': 'Palos Verdes Estates', 'county_fips': '06037', 'geo_precision': 'rooftop', 22 | 'block_id': '060376703241005', 'zipcode': '90274', 23 | 'address_full': '43 Valmonte Plz Palos Verdes Estates CA 90274', 24 | 'state': 'CA', 'zipcode_plus4': '1444', 'address': '43 Valmonte Plz', 25 | 'lat': 33.79814, 'lng': -118.36455, 26 | 'slug': '43-Valmonte-Plz-Palos-Verdes-Estates-CA-90274', 'unit': None 27 | }, 28 | 'meta': 'Test Meta' 29 | } 30 | 31 | self.prop = Property.create_from_json(self.test_json) 32 | 33 | def test_create_from_json(self): 34 | self.assertEqual(self.prop.address, "43 Valmonte Plz") 35 | self.assertEqual(self.prop.county_fips, "06037") 36 | self.assertEqual(self.prop.geo_precision, "rooftop") 37 | self.assertEqual(self.prop.block_id, "060376703241005") 38 | self.assertEqual(self.prop.zipcode, "90274") 39 | self.assertEqual(self.prop.address_full, "43 Valmonte Plz Palos Verdes Estates CA 90274") 40 | self.assertEqual(self.prop.state, "CA") 41 | self.assertEqual(self.prop.zipcode_plus4, "1444") 42 | self.assertEqual(self.prop.city, "Palos Verdes Estates") 43 | self.assertEqual(self.prop.lat, 33.79814) 44 | self.assertEqual(self.prop.lng, -118.36455) 45 | self.assertEqual(self.prop.slug, "43-Valmonte-Plz-Palos-Verdes-Estates-CA-90274") 46 | self.assertEqual(self.prop.unit, None) 47 | self.assertEqual(self.prop.meta, 'Test Meta') 48 | self.assertEqual(len(self.prop.component_results), 1) 49 | self.assertEqual(self.prop.component_results[0].api_code, 0) 50 | self.assertEqual(self.prop.component_results[0].api_code_description, 'ok') 51 | self.assertEqual(self.prop.component_results[0].json_data, { 52 | 'price_pr': 2938.0, 'price_lwr': 2160.0, 'price_mean': 2296.0, 'fsd': 0.17 53 | }) 54 | 55 | def test_create_from_json_with_multiple_components(self): 56 | test_json2 = { 57 | 'property/value': { 58 | 'api_code_description': 'ok', 59 | 'api_code': 0, 60 | 'result': { 61 | 'price_pr': 2938.0, 'price_lwr': 2160.0, 'price_mean': 2296.0, 'fsd': 0.17 62 | } 63 | }, 64 | 'property/census': { 65 | 'api_code_description': 'ok', 66 | 'api_code': 0, 67 | 'result': 'dummy data' 68 | }, 69 | 'address_info': { 70 | 'city': 'Palos Verdes Estates', 'county_fips': '06037', 'geo_precision': 'rooftop', 71 | 'block_id': '060376703241005', 'zipcode': '90274', 72 | 'address_full': '43 Valmonte Plz Palos Verdes Estates CA 90274', 73 | 'state': 'CA', 'zipcode_plus4': '1444', 'address': '43 Valmonte Plz', 74 | 'lat': 33.79814, 'lng': -118.36455, 75 | 'slug': '43-Valmonte-Plz-Palos-Verdes-Estates-CA-90274', 'unit': None 76 | }, 77 | 'meta': 'Test Meta' 78 | } 79 | 80 | prop2 = Property.create_from_json(test_json2) 81 | 82 | self.assertEqual(prop2.address, "43 Valmonte Plz") 83 | self.assertEqual(len(prop2.component_results), 2) 84 | value_result = next( 85 | (cr for cr in prop2.component_results if cr.component_name == "property/value"), 86 | None 87 | ) 88 | self.assertIsNotNone(value_result) 89 | census_result = next( 90 | (cr for cr in prop2.component_results if cr.component_name == "property/census"), 91 | None 92 | ) 93 | self.assertEqual(census_result.json_data, "dummy data") 94 | 95 | def test_has_error(self): 96 | self.assertFalse(self.prop.has_error()) 97 | 98 | def test_has_error_with_error(self): 99 | self.test_json['property/value']['api_code'] = 1001 100 | prop2 = Property.create_from_json(self.test_json) 101 | self.assertTrue(prop2.has_error()) 102 | 103 | def test_get_errors(self): 104 | self.assertEqual(self.prop.get_errors(), []) 105 | 106 | def test_get_errors_with_errors(self): 107 | self.test_json['property/value']['api_code'] = 1001 108 | self.test_json['property/value']['api_code_description'] = "test error" 109 | prop2 = Property.create_from_json(self.test_json) 110 | self.assertEqual(prop2.get_errors(), [{"property/value": "test error"}]) 111 | 112 | 113 | class BlockTestCase(unittest.TestCase): 114 | def setUp(self): 115 | self.test_json = { 116 | 'block/value_ts': { 117 | 'api_code_description': 'ok', 118 | 'api_code': 0, 119 | 'result': { 120 | 'value_sqft_median': 303.36 121 | } 122 | }, 123 | 'block_info': { 124 | 'property_type': 'SFD', 125 | 'block_id': '060376703241005' 126 | }, 127 | 'meta': 'Test Meta' 128 | } 129 | 130 | self.block = Block.create_from_json(self.test_json) 131 | 132 | def test_create_from_json(self): 133 | self.assertEqual(self.block.block_id, "060376703241005") 134 | self.assertEqual(self.block.property_type, "SFD") 135 | self.assertEqual(self.block.num_bins, None) 136 | self.assertEqual(self.block.meta, "Test Meta") 137 | self.assertEqual(len(self.block.component_results), 1) 138 | self.assertEqual(self.block.component_results[0].api_code, 0) 139 | self.assertEqual(self.block.component_results[0].api_code_description, 'ok') 140 | self.assertEqual(self.block.component_results[0].json_data, {'value_sqft_median': 303.36}) 141 | 142 | def test_create_from_json_with_num_bins(self): 143 | json = self.test_json.copy() 144 | json["block_info"] = { 145 | "block_id": "060376703241005", 146 | "num_bins": "5" 147 | } 148 | self.block = Block.create_from_json(json) 149 | self.assertEqual(self.block.block_id, "060376703241005") 150 | self.assertEqual(self.block.property_type, None) 151 | self.assertEqual(self.block.num_bins, "5") 152 | 153 | def test_create_from_json_with_multiple_components(self): 154 | test_json2 = { 155 | 'block/value_ts': { 156 | 'api_code_description': 'ok', 157 | 'api_code': 0, 158 | 'result': { 159 | 'value_sqft_median': 303.36 160 | } 161 | }, 162 | 'block/histogram_beds': { 163 | 'api_code_description': 'ok', 164 | 'api_code': 0, 165 | 'result': 'dummy data' 166 | }, 167 | 'block_info': { 168 | 'property_type': 'SFD', 169 | 'block_id': '060376703241005' 170 | }, 171 | 'meta': 'Test Meta' 172 | } 173 | 174 | block2 = Block.create_from_json(test_json2) 175 | 176 | self.assertEqual(block2.block_id, "060376703241005") 177 | self.assertEqual(len(block2.component_results), 2) 178 | result1 = next( 179 | (cr for cr in block2.component_results if cr.component_name == "block/value_ts"), 180 | None 181 | ) 182 | self.assertIsNotNone(result1) 183 | result2 = next( 184 | (cr for cr in block2.component_results if cr.component_name == "block/histogram_beds"), 185 | None 186 | ) 187 | self.assertEqual(result2.json_data, "dummy data") 188 | 189 | def test_has_error(self): 190 | self.assertFalse(self.block.has_error()) 191 | 192 | def test_has_error_with_error(self): 193 | self.test_json['block/value_ts']['api_code'] = 1001 194 | block2 = Block.create_from_json(self.test_json) 195 | self.assertTrue(block2.has_error()) 196 | 197 | def test_get_errors(self): 198 | self.assertEqual(self.block.get_errors(), []) 199 | 200 | def test_get_errors_with_errors(self): 201 | self.test_json['block/value_ts']['api_code'] = 1001 202 | self.test_json['block/value_ts']['api_code_description'] = "test error" 203 | block2 = Block.create_from_json(self.test_json) 204 | self.assertEqual(block2.get_errors(), [{"block/value_ts": "test error"}]) 205 | 206 | 207 | class ZipTestCase(unittest.TestCase): 208 | def setUp(self): 209 | self.test_json = { 210 | 'zip/details': { 211 | 'api_code_description': 'ok', 212 | 'api_code': 0, 213 | 'result': 'some result' 214 | }, 215 | 'zipcode_info': { 216 | 'zipcode': '90274' 217 | }, 218 | 'meta': 'Test Meta' 219 | } 220 | 221 | self.zip = ZipCode.create_from_json(self.test_json) 222 | 223 | def test_create_from_json(self): 224 | self.assertEqual(self.zip.zipcode, "90274") 225 | self.assertEqual(self.zip.meta, "Test Meta") 226 | self.assertEqual(len(self.zip.component_results), 1) 227 | self.assertEqual(self.zip.component_results[0].api_code, 0) 228 | self.assertEqual(self.zip.component_results[0].api_code_description, 'ok') 229 | self.assertEqual(self.zip.component_results[0].json_data, 'some result') 230 | 231 | def test_create_from_json_with_multiple_components(self): 232 | test_json2 = { 233 | 'zip/details': { 234 | 'api_code_description': 'ok', 235 | 'api_code': 0, 236 | 'result': 'details result' 237 | }, 238 | 'zip/volatility': { 239 | 'api_code_description': 'ok', 240 | 'api_code': 0, 241 | 'result': 'dummy data' 242 | }, 243 | 'zipcode_info': { 244 | 'zipcode': '90274', 245 | }, 246 | 'meta': 'Test Meta' 247 | } 248 | 249 | zip2 = ZipCode.create_from_json(test_json2) 250 | 251 | self.assertEqual(zip2.zipcode, "90274") 252 | self.assertEqual(len(zip2.component_results), 2) 253 | result1 = next( 254 | (cr for cr in zip2.component_results if cr.component_name == "zip/details"), 255 | None 256 | ) 257 | self.assertEqual(result1.json_data, "details result") 258 | result2 = next( 259 | (cr for cr in zip2.component_results if cr.component_name == "zip/volatility"), 260 | None 261 | ) 262 | self.assertEqual(result2.json_data, "dummy data") 263 | 264 | def test_has_error(self): 265 | self.assertFalse(self.zip.has_error()) 266 | 267 | def test_has_error_with_error(self): 268 | self.test_json['zip/details']['api_code'] = 1001 269 | zip2 = ZipCode.create_from_json(self.test_json) 270 | self.assertTrue(zip2.has_error()) 271 | 272 | def test_get_errors(self): 273 | self.assertEqual(self.zip.get_errors(), []) 274 | 275 | def test_get_errors_with_errors(self): 276 | self.test_json['zip/details']['api_code'] = 1001 277 | self.test_json['zip/details']['api_code_description'] = "test error" 278 | zip2 = ZipCode.create_from_json(self.test_json) 279 | self.assertEqual(zip2.get_errors(), [{"zip/details": "test error"}]) 280 | 281 | 282 | class MsaTestCase(unittest.TestCase): 283 | def setUp(self): 284 | self.test_json = { 285 | 'msa/details': { 286 | 'api_code_description': 'ok', 287 | 'api_code': 0, 288 | 'result': 'some result' 289 | }, 290 | 'msa_info': { 291 | 'msa': '41860' 292 | }, 293 | 'meta': 'Test Meta' 294 | } 295 | 296 | self.msa = Msa.create_from_json(self.test_json) 297 | 298 | def test_create_from_json(self): 299 | self.assertEqual(self.msa.msa, "41860") 300 | self.assertEqual(self.msa.meta, "Test Meta") 301 | self.assertEqual(len(self.msa.component_results), 1) 302 | self.assertEqual(self.msa.component_results[0].api_code, 0) 303 | self.assertEqual(self.msa.component_results[0].api_code_description, 'ok') 304 | self.assertEqual(self.msa.component_results[0].json_data, 'some result') 305 | 306 | def test_create_from_json_with_multiple_components(self): 307 | test_json2 = { 308 | 'msa/details': { 309 | 'api_code_description': 'ok', 310 | 'api_code': 0, 311 | 'result': 'details result' 312 | }, 313 | 'msa/hpi_ts': { 314 | 'api_code_description': 'ok', 315 | 'api_code': 0, 316 | 'result': 'dummy data' 317 | }, 318 | 'msa_info': { 319 | 'msa': '41860', 320 | }, 321 | 'meta': 'Test Meta' 322 | } 323 | 324 | msa2 = Msa.create_from_json(test_json2) 325 | 326 | self.assertEqual(msa2.msa, "41860") 327 | self.assertEqual(len(msa2.component_results), 2) 328 | result1 = next( 329 | (cr for cr in msa2.component_results if cr.component_name == "msa/details"), 330 | None 331 | ) 332 | self.assertEqual(result1.json_data, "details result") 333 | result2 = next( 334 | (cr for cr in msa2.component_results if cr.component_name == "msa/hpi_ts"), 335 | None 336 | ) 337 | self.assertEqual(result2.json_data, "dummy data") 338 | 339 | def test_has_error(self): 340 | self.assertFalse(self.msa.has_error()) 341 | 342 | def test_has_error_with_error(self): 343 | self.test_json['msa/details']['api_code'] = 1001 344 | msa2 = Msa.create_from_json(self.test_json) 345 | self.assertTrue(msa2.has_error()) 346 | 347 | def test_get_errors(self): 348 | self.assertEqual(self.msa.get_errors(), []) 349 | 350 | def test_get_errors_with_errors(self): 351 | self.test_json['msa/details']['api_code'] = 1001 352 | self.test_json['msa/details']['api_code_description'] = "test error" 353 | msa2 = Msa.create_from_json(self.test_json) 354 | self.assertEqual(msa2.get_errors(), [{"msa/details": "test error"}]) 355 | 356 | 357 | if __name__ == "__main__": 358 | unittest.main() 359 | -------------------------------------------------------------------------------- /tests/test_response.py: -------------------------------------------------------------------------------- 1 | # pylint: disable=missing-docstring 2 | 3 | import unittest 4 | import requests_mock 5 | from housecanary.apiclient import ApiClient 6 | from housecanary.object import Property 7 | from housecanary.object import Block 8 | from housecanary.object import ZipCode 9 | from housecanary.object import Msa 10 | 11 | 12 | @requests_mock.Mocker() 13 | class PropertyResponseTestCase(unittest.TestCase): 14 | def setUp(self): 15 | self.client = ApiClient() 16 | self.test_data = [{"address": "43 Valmonte Plaza", "zipcode": "90274"}] 17 | self.headers = {'content-type': 'application/json', 'X-RateLimit-Limit': '5000', 'X-RateLimit-Reset': '1491920221', 'X-RateLimit-Period': '60', 'X-RateLimit-Remaining': '4999'} 18 | self.response = [{'property/value': {'api_code_description': 'ok', 'result': {'value': {'price_upr': 1780318, 'fsd': 0.0836871, 'price_mean': 1642834, 'price_lwr': 1505350}}, 'api_code': 0}, 'address_info': {'state': 'CA', 'address_full': '43 Valmonte Plz Palos Verdes Estates CA 90274', 'block_id': '060376703241005', 'slug': '43-Valmonte-Plz-Palos-Verdes-Estates-CA-90274', 'msa': '31080', 'zipcode': '90274', 'county_fips': '06037', 'city': 'Palos Verdes Estates', 'geo_precision': 'rooftop', 'unit': None, 'address': '43 Valmonte Plz', 'lat': 33.79814, 'zipcode_plus4': '1444', 'metrodiv': '31084', 'lng': -118.36455}}] 19 | self.multi_property_response = [{'property/value': {'api_code_description': 'ok', 'result': {'value': {'price_upr': 1780318, 'fsd': 0.0836871, 'price_mean': 1642834, 'price_lwr': 1505350}}, 'api_code': 0}, 'address_info': {'state': 'CA', 'address_full': '43 Valmonte Plz Palos Verdes Estates CA 90274', 'block_id': '060376703241005', 'slug': '43-Valmonte-Plz-Palos-Verdes-Estates-CA-90274', 'msa': '31080', 'zipcode': '90274', 'county_fips': '06037', 'city': 'Palos Verdes Estates', 'geo_precision': 'rooftop', 'unit': None, 'address': '43 Valmonte Plz', 'lat': 33.79814, 'zipcode_plus4': '1444', 'metrodiv': '31084', 'lng': -118.36455}}, {'property/value': {'api_code_description': 'no content', 'result': None, 'api_code': 204}, 'address_info': {'state': None, 'address_full': None, 'block_id': None, 'slug': None, 'msa': None, 'zipcode': None, 'county_fips': None, 'city': None, 'geo_precision': 'unknown', 'unit': None, 'address': None, 'lat': None, 'zipcode_plus4': None, 'metrodiv': None, 'lng': None}}] 20 | self.error_response = [{'property/value': {'api_code_description': 'no content', 'result': None, 'api_code': 204}, 'address_info': {'state': None, 'address_full': None, 'block_id': None, 'slug': None, 'msa': None, 'zipcode': None, 'county_fips': None, 'city': None, 'geo_precision': 'unknown', 'unit': None, 'address': None, 'lat': None, 'zipcode_plus4': None, 'metrodiv': None, 'lng': None}}] 21 | 22 | def test_endpoint_name(self, mock): 23 | mock.get("/v2/property/value", headers=self.headers, json=self.response) 24 | response = self.client.fetch("property/value", self.test_data) 25 | self.assertEqual(response.endpoint_name, "property/value") 26 | 27 | def test_json(self, mock): 28 | mock.get("/v2/property/value", headers=self.headers, json=self.response) 29 | response = self.client.fetch("property/value", self.test_data) 30 | self.assertTrue(isinstance(response.json(), list)) 31 | 32 | def test_response(self, mock): 33 | mock.get("/v2/property/value", headers=self.headers, json=self.response) 34 | response = self.client.fetch("property/value", self.test_data) 35 | self.assertIsNotNone(response.response) 36 | 37 | def test_get_object_errors(self, mock): 38 | mock.get("/v2/property/value", headers=self.headers, json=self.response) 39 | response = self.client.fetch("property/value", self.test_data) 40 | self.assertEqual(response.get_object_errors(), []) 41 | 42 | def test_get_object_errors_with_error(self, mock): 43 | mock.get("/v2/property/value", headers=self.headers, json=self.error_response) 44 | self.test_data[0]["zipcode"] = "00000" 45 | response = self.client.fetch("property/value", self.test_data) 46 | self.assertEqual(len(response.get_object_errors()), 1) 47 | 48 | def test_has_object_error(self, mock): 49 | mock.get("/v2/property/value", headers=self.headers, json=self.response) 50 | response = self.client.fetch("property/value", self.test_data) 51 | self.assertFalse(response.has_object_error()) 52 | 53 | def test_has_object_error_with_error(self, mock): 54 | mock.get("/v2/property/value", headers=self.headers, json=self.error_response) 55 | self.test_data[0]["zipcode"] = "00000" 56 | response = self.client.fetch("property/value", self.test_data) 57 | self.assertTrue(response.has_object_error()) 58 | 59 | def test_properties(self, mock): 60 | mock.get("/v2/property/value", headers=self.headers, json=self.response) 61 | response = self.client.fetch("property/value", self.test_data) 62 | self.assertTrue(len(response.properties()), 1) 63 | self.assertTrue(isinstance(response.properties()[0], Property)) 64 | 65 | def test_properties_with_multiple(self, mock): 66 | mock.post("/v2/property/value", headers=self.headers, json=self.multi_property_response) 67 | self.test_data.append({"address": "85 Clay St", "zipcode": "02140"}) 68 | response = self.client.fetch("property/value", self.test_data) 69 | self.assertTrue(len(response.properties()), 2) 70 | self.assertTrue(isinstance(response.properties()[0], Property)) 71 | self.assertTrue(isinstance(response.properties()[1], Property)) 72 | 73 | def test_response_rate_limits(self, mock): 74 | mock.get("/v2/property/value", headers=self.headers, json=self.response) 75 | response = self.client.fetch("property/value", self.test_data) 76 | self.assertTrue(isinstance(response.rate_limits, list)) 77 | self.assertTrue(isinstance(response.rate_limits[0]['period'], str)) 78 | 79 | 80 | @requests_mock.Mocker() 81 | class BlockResponseTestCase(unittest.TestCase): 82 | def setUp(self): 83 | self.client = ApiClient() 84 | self.test_data = [{"block_id": "060376703241005"}] 85 | self.headers = {'content-type': 'application/json', 'X-RateLimit-Limit': '5000', 'X-RateLimit-Reset': '1491920221', 'X-RateLimit-Period': '60', 'X-RateLimit-Remaining': '4999'} 86 | self.response = [{'block_info': {'block_id': '060376703241005'}, 'block/value_ts': {'api_code_description': 'ok', 'result': {'time_series': [{'value_sqft_median': 296.89, 'month': '1994-02-01', 'value_median': 513529}], 'property_type': 'SFD'}, 'api_code': 0}}] 87 | self.response_multi = [{'block_info': {'block_id': '060376703241005'}, 'block/value_ts': {'api_code_description': 'ok', 'result': {'time_series': [{'value_sqft_median': 296.89, 'month': '1994-02-01', 'value_median': 513529}], 'property_type': 'SFD'}, 'api_code': 0}}, {'block_info': {'block_id': '160376703241005'}, 'block/value_ts': {'api_code_description': 'ok', 'result': {'time_series': [{'value_sqft_median': 296.89, 'month': '1994-02-01', 'value_median': 513529}], 'property_type': 'SFD'}, 'api_code': 0}}] 88 | 89 | def test_blocks(self, mock): 90 | mock.get("/v2/block/value_ts", headers=self.headers, json=self.response) 91 | response = self.client.fetch("block/value_ts", self.test_data) 92 | self.assertTrue(len(response.blocks()), 1) 93 | self.assertTrue(isinstance(response.blocks()[0], Block)) 94 | 95 | def test_blocks_with_multiple(self, mock): 96 | mock.post("/v2/block/value_ts", headers=self.headers, json=self.response_multi) 97 | self.test_data.append({"block_id": "160376703241005"}) 98 | response = self.client.fetch("block/value_ts", self.test_data) 99 | self.assertTrue(len(response.blocks()), 2) 100 | self.assertTrue(isinstance(response.blocks()[0], Block)) 101 | self.assertTrue(isinstance(response.blocks()[1], Block)) 102 | 103 | 104 | @requests_mock.Mocker() 105 | class ZipCodeResponseTestCase(unittest.TestCase): 106 | def setUp(self): 107 | self.client = ApiClient() 108 | self.test_data = [{"zipcode": "90274"}] 109 | self.headers = {'content-type': 'application/json', 'X-RateLimit-Limit': '5000', 'X-RateLimit-Reset': '1491920221', 'X-RateLimit-Period': '60', 'X-RateLimit-Remaining': '4999'} 110 | self.response = [{'zip/details': {'api_code_description': 'ok', 'result': {'single_family': {'inventory_total': 78.538, 'price_median': 2748899.085, 'estimated_sales_total': None, 'days_on_market_median': 116.132, 'months_of_inventory_median': None, 'market_action_median': 62.34}, 'multi_family': {'inventory_total': None, 'price_median': None, 'estimated_sales_total': None, 'days_on_market_median': None, 'months_of_inventory_median': None, 'market_action_median': None}}, 'api_code': 0}, 'zipcode_info': {'zipcode': '90274'}}] 111 | self.response_multi = [{'zip/details': {'api_code_description': 'ok', 'result': {'single_family': {'inventory_total': 78.538, 'price_median': 2748899.085, 'estimated_sales_total': None, 'days_on_market_median': 116.132, 'months_of_inventory_median': None, 'market_action_median': 62.34}, 'multi_family': {'inventory_total': None, 'price_median': None, 'estimated_sales_total': None, 'days_on_market_median': None, 'months_of_inventory_median': None, 'market_action_median': None}}, 'api_code': 0}, 'zipcode_info': {'zipcode': '90274'}}, {'zip/details': {'api_code_description': 'ok', 'result': {'single_family': {'inventory_total': 28.462, 'price_median': 453311.462, 'estimated_sales_total': 24.908, 'days_on_market_median': 46.308, 'months_of_inventory_median': 1.143, 'market_action_median': 79.47}, 'multi_family': {'inventory_total': 8.923, 'price_median': 236076.923, 'estimated_sales_total': 6.73, 'days_on_market_median': 67.038, 'months_of_inventory_median': 1.326, 'market_action_median': 81.33}}, 'api_code': 0}, 'zipcode_info': {'zipcode': '01960'}}] 112 | 113 | def test_zipcodes(self, mock): 114 | mock.get("/v2/zip/details", headers=self.headers, json=self.response) 115 | response = self.client.fetch("zip/details", self.test_data) 116 | self.assertTrue(len(response.zipcodes()), 1) 117 | self.assertTrue(isinstance(response.zipcodes()[0], ZipCode)) 118 | 119 | def test_zipcodes_with_multiple(self, mock): 120 | mock.post("/v2/zip/details", headers=self.headers, json=self.response_multi) 121 | self.test_data.append({"zipcode": "01960"}) 122 | response = self.client.fetch("zip/details", self.test_data) 123 | self.assertTrue(len(response.zipcodes()), 2) 124 | self.assertTrue(isinstance(response.zipcodes()[0], ZipCode)) 125 | self.assertTrue(isinstance(response.zipcodes()[1], ZipCode)) 126 | 127 | 128 | @requests_mock.Mocker() 129 | class MsaResponseTestCase(unittest.TestCase): 130 | def setUp(self): 131 | self.client = ApiClient() 132 | self.test_data = [{"msa": "41860"}] 133 | self.headers = {'content-type': 'application/json', 'X-RateLimit-Limit': '5000', 'X-RateLimit-Reset': '1491920221', 'X-RateLimit-Period': '60', 'X-RateLimit-Remaining': '4999'} 134 | self.response = [{'msa_info': {'msa': '41860', 'msa_name': 'San Francisco-Oakland-Hayward, CA'}, 'msa/details': {'api_code_description': 'ok', 'result': {'returns_5': 0.8724, 'cagr_1': 0.0673, 'cagr_5': 0.1336, 'max_12mo_loss': -0.210943, 'cagr_10': 0.0183, 'returns_1': 0.0673, 'risk_12mo_loss': 0.080268, 'cagr_20': 0.0658, 'returns_10': 0.1985}, 'api_code': 0}}] 135 | self.response_multi = [{'msa_info': {'msa': '41860', 'msa_name': 'San Francisco-Oakland-Hayward, CA'}, 'msa/details': {'api_code_description': 'ok', 'result': {'returns_5': 0.8724, 'cagr_1': 0.0673, 'cagr_5': 0.1336, 'max_12mo_loss': -0.210943, 'cagr_10': 0.0183, 'returns_1': 0.0673, 'risk_12mo_loss': 0.080268, 'cagr_20': 0.0658, 'returns_10': 0.1985}, 'api_code': 0}}, {'msa_info': {'msa': '41861', 'msa_name': 'San Francisco-Oakland-Hayward, CA'}, 'msa/details': {'api_code_description': 'ok', 'result': {'returns_5': 0.8724, 'cagr_1': 0.0673, 'cagr_5': 0.1336, 'max_12mo_loss': -0.210943, 'cagr_10': 0.0183, 'returns_1': 0.0673, 'risk_12mo_loss': 0.080268, 'cagr_20': 0.0658, 'returns_10': 0.1985}, 'api_code': 0}}] 136 | 137 | def test_msas(self, mock): 138 | mock.get("/v2/msa/details", headers=self.headers, json=self.response) 139 | response = self.client.fetch("msa/details", self.test_data) 140 | self.assertTrue(len(response.msas()), 1) 141 | self.assertTrue(isinstance(response.msas()[0], Msa)) 142 | 143 | def test_msas_with_multiple(self, mock): 144 | mock.post("/v2/msa/details", headers=self.headers, json=self.response_multi) 145 | self.test_data.append({"msa": "41861"}) 146 | response = self.client.fetch("msa/details", self.test_data) 147 | self.assertTrue(len(response.msas()), 2) 148 | self.assertTrue(isinstance(response.msas()[0], Msa)) 149 | self.assertTrue(isinstance(response.msas()[1], Msa)) 150 | 151 | 152 | if __name__ == "__main__": 153 | unittest.main() 154 | -------------------------------------------------------------------------------- /tests/test_utilities.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | from housecanary import utilities 3 | 4 | 5 | class UtilitiesTestCase(unittest.TestCase): 6 | """Tests for Utilities""" 7 | 8 | def test_get_readable_time_string(self): 9 | self.assertEqual('30 Seconds', utilities.get_readable_time_string(30)) 10 | self.assertEqual('1 Minute', utilities.get_readable_time_string(60)) 11 | self.assertEqual('5 Minutes', utilities.get_readable_time_string(300)) 12 | self.assertEqual('5 Minutes 30 Seconds', utilities.get_readable_time_string(330)) 13 | self.assertEqual('1 Hour 1 Minute 1 Second', utilities.get_readable_time_string(3661)) 14 | self.assertEqual('2 Hours', utilities.get_readable_time_string(7200)) 15 | self.assertEqual('1 Day 2 Hours', utilities.get_readable_time_string(93600)) 16 | self.assertEqual('2 Days', utilities.get_readable_time_string(172800)) 17 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [flake8] 2 | max-line-length = 100 --------------------------------------------------------------------------------