├── CODE_OF_CONDUCT.md ├── LICENSE.txt ├── README.md ├── aprsdb.py ├── aprsdb_creation.sql ├── aprsgps.py ├── db_access_privs.sql ├── db_roles.sql ├── enable_postgis.sql ├── generic_aprsdb.conf └── rm_tables.sql /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as 6 | contributors and maintainers pledge to making participation in our project and 7 | our community a harassment-free experience for everyone, regardless of age, body 8 | size, disability, ethnicity, sex characteristics, gender identity and expression, 9 | level of experience, education, socio-economic status, nationality, personal 10 | appearance, race, religion, or sexual identity and orientation. 11 | 12 | ## Our Standards 13 | 14 | Examples of behavior that contributes to creating a positive environment 15 | include: 16 | 17 | * Using welcoming and inclusive language 18 | * Being respectful of differing viewpoints and experiences 19 | * Gracefully accepting constructive criticism 20 | * Focusing on what is best for the community 21 | * Showing empathy towards other community members 22 | 23 | Examples of unacceptable behavior by participants include: 24 | 25 | * The use of sexualized language or imagery and unwelcome sexual attention or 26 | advances 27 | * Trolling, insulting/derogatory comments, and personal or political attacks 28 | * Public or private harassment 29 | * Publishing others' private information, such as a physical or electronic 30 | address, without explicit permission 31 | * Other conduct which could reasonably be considered inappropriate in a 32 | professional setting 33 | 34 | ## Our Responsibilities 35 | 36 | Project maintainers are responsible for clarifying the standards of acceptable 37 | behavior and are expected to take appropriate and fair corrective action in 38 | response to any instances of unacceptable behavior. 39 | 40 | Project maintainers have the right and responsibility to remove, edit, or 41 | reject comments, commits, code, wiki edits, issues, and other contributions 42 | that are not aligned to this Code of Conduct, or to ban temporarily or 43 | permanently any contributor for other behaviors that they deem inappropriate, 44 | threatening, offensive, or harmful. 45 | 46 | ## Scope 47 | 48 | This Code of Conduct applies both within project spaces and in public spaces 49 | when an individual is representing the project or its community. Examples of 50 | representing a project or community include using an official project e-mail 51 | address, posting via an official social media account, or acting as an appointed 52 | representative at an online or offline event. Representation of a project may be 53 | further defined and clarified by project maintainers. 54 | 55 | ## Enforcement 56 | 57 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 58 | reported by contacting the project team. All 59 | complaints will be reviewed and investigated and will result in a response that 60 | is deemed necessary and appropriate to the circumstances. The project team is 61 | obligated to maintain confidentiality with regard to the reporter of an incident. 62 | Further details of specific enforcement policies may be posted separately. 63 | 64 | Project maintainers who do not follow or enforce the Code of Conduct in good 65 | faith may face temporary or permanent repercussions as determined by other 66 | members of the project's leadership. 67 | 68 | ## Attribution 69 | 70 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, 71 | available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html 72 | 73 | [homepage]: https://www.contributor-covenant.org 74 | 75 | For answers to common questions about this code of conduct, see 76 | https://www.contributor-covenant.org/faq 77 | 78 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 3, 29 June 2007 3 | 4 | Copyright (C) 2007 Free Software Foundation, Inc. 5 | Everyone is permitted to copy and distribute verbatim copies 6 | of this license document, but changing it is not allowed. 7 | 8 | Preamble 9 | 10 | The GNU General Public License is a free, copyleft license for 11 | software and other kinds of works. 12 | 13 | The licenses for most software and other practical works are designed 14 | to take away your freedom to share and change the works. By contrast, 15 | the GNU General Public License is intended to guarantee your freedom to 16 | share and change all versions of a program--to make sure it remains free 17 | software for all its users. We, the Free Software Foundation, use the 18 | GNU General Public License for most of our software; it applies also to 19 | any other work released this way by its authors. You can apply it to 20 | your programs, too. 21 | 22 | When we speak of free software, we are referring to freedom, not 23 | price. Our General Public Licenses are designed to make sure that you 24 | have the freedom to distribute copies of free software (and charge for 25 | them if you wish), that you receive source code or can get it if you 26 | want it, that you can change the software or use pieces of it in new 27 | free programs, and that you know you can do these things. 28 | 29 | To protect your rights, we need to prevent others from denying you 30 | these rights or asking you to surrender the rights. Therefore, you have 31 | certain responsibilities if you distribute copies of the software, or if 32 | you modify it: responsibilities to respect the freedom of others. 33 | 34 | For example, if you distribute copies of such a program, whether 35 | gratis or for a fee, you must pass on to the recipients the same 36 | freedoms that you received. You must make sure that they, too, receive 37 | or can get the source code. And you must show them these terms so they 38 | know their rights. 39 | 40 | Developers that use the GNU GPL protect your rights with two steps: 41 | (1) assert copyright on the software, and (2) offer you this License 42 | giving you legal permission to copy, distribute and/or modify it. 43 | 44 | For the developers' and authors' protection, the GPL clearly explains 45 | that there is no warranty for this free software. For both users' and 46 | authors' sake, the GPL requires that modified versions be marked as 47 | changed, so that their problems will not be attributed erroneously to 48 | authors of previous versions. 49 | 50 | Some devices are designed to deny users access to install or run 51 | modified versions of the software inside them, although the manufacturer 52 | can do so. This is fundamentally incompatible with the aim of 53 | protecting users' freedom to change the software. The systematic 54 | pattern of such abuse occurs in the area of products for individuals to 55 | use, which is precisely where it is most unacceptable. Therefore, we 56 | have designed this version of the GPL to prohibit the practice for those 57 | products. If such problems arise substantially in other domains, we 58 | stand ready to extend this provision to those domains in future versions 59 | of the GPL, as needed to protect the freedom of users. 60 | 61 | Finally, every program is threatened constantly by software patents. 62 | States should not allow patents to restrict development and use of 63 | software on general-purpose computers, but in those that do, we wish to 64 | avoid the special danger that patents applied to a free program could 65 | make it effectively proprietary. To prevent this, the GPL assures that 66 | patents cannot be used to render the program non-free. 67 | 68 | The precise terms and conditions for copying, distribution and 69 | modification follow. 70 | 71 | TERMS AND CONDITIONS 72 | 73 | 0. Definitions. 74 | 75 | "This License" refers to version 3 of the GNU General Public License. 76 | 77 | "Copyright" also means copyright-like laws that apply to other kinds of 78 | works, such as semiconductor masks. 79 | 80 | "The Program" refers to any copyrightable work licensed under this 81 | License. Each licensee is addressed as "you". "Licensees" and 82 | "recipients" may be individuals or organizations. 83 | 84 | To "modify" a work means to copy from or adapt all or part of the work 85 | in a fashion requiring copyright permission, other than the making of an 86 | exact copy. The resulting work is called a "modified version" of the 87 | earlier work or a work "based on" the earlier work. 88 | 89 | A "covered work" means either the unmodified Program or a work based 90 | on the Program. 91 | 92 | To "propagate" a work means to do anything with it that, without 93 | permission, would make you directly or secondarily liable for 94 | infringement under applicable copyright law, except executing it on a 95 | computer or modifying a private copy. Propagation includes copying, 96 | distribution (with or without modification), making available to the 97 | public, and in some countries other activities as well. 98 | 99 | To "convey" a work means any kind of propagation that enables other 100 | parties to make or receive copies. Mere interaction with a user through 101 | a computer network, with no transfer of a copy, is not conveying. 102 | 103 | An interactive user interface displays "Appropriate Legal Notices" 104 | to the extent that it includes a convenient and prominently visible 105 | feature that (1) displays an appropriate copyright notice, and (2) 106 | tells the user that there is no warranty for the work (except to the 107 | extent that warranties are provided), that licensees may convey the 108 | work under this License, and how to view a copy of this License. If 109 | the interface presents a list of user commands or options, such as a 110 | menu, a prominent item in the list meets this criterion. 111 | 112 | 1. Source Code. 113 | 114 | The "source code" for a work means the preferred form of the work 115 | for making modifications to it. "Object code" means any non-source 116 | form of a work. 117 | 118 | A "Standard Interface" means an interface that either is an official 119 | standard defined by a recognized standards body, or, in the case of 120 | interfaces specified for a particular programming language, one that 121 | is widely used among developers working in that language. 122 | 123 | The "System Libraries" of an executable work include anything, other 124 | than the work as a whole, that (a) is included in the normal form of 125 | packaging a Major Component, but which is not part of that Major 126 | Component, and (b) serves only to enable use of the work with that 127 | Major Component, or to implement a Standard Interface for which an 128 | implementation is available to the public in source code form. A 129 | "Major Component", in this context, means a major essential component 130 | (kernel, window system, and so on) of the specific operating system 131 | (if any) on which the executable work runs, or a compiler used to 132 | produce the work, or an object code interpreter used to run it. 133 | 134 | The "Corresponding Source" for a work in object code form means all 135 | the source code needed to generate, install, and (for an executable 136 | work) run the object code and to modify the work, including scripts to 137 | control those activities. However, it does not include the work's 138 | System Libraries, or general-purpose tools or generally available free 139 | programs which are used unmodified in performing those activities but 140 | which are not part of the work. For example, Corresponding Source 141 | includes interface definition files associated with source files for 142 | the work, and the source code for shared libraries and dynamically 143 | linked subprograms that the work is specifically designed to require, 144 | such as by intimate data communication or control flow between those 145 | subprograms and other parts of the work. 146 | 147 | The Corresponding Source need not include anything that users 148 | can regenerate automatically from other parts of the Corresponding 149 | Source. 150 | 151 | The Corresponding Source for a work in source code form is that 152 | same work. 153 | 154 | 2. Basic Permissions. 155 | 156 | All rights granted under this License are granted for the term of 157 | copyright on the Program, and are irrevocable provided the stated 158 | conditions are met. This License explicitly affirms your unlimited 159 | permission to run the unmodified Program. The output from running a 160 | covered work is covered by this License only if the output, given its 161 | content, constitutes a covered work. This License acknowledges your 162 | rights of fair use or other equivalent, as provided by copyright law. 163 | 164 | You may make, run and propagate covered works that you do not 165 | convey, without conditions so long as your license otherwise remains 166 | in force. You may convey covered works to others for the sole purpose 167 | of having them make modifications exclusively for you, or provide you 168 | with facilities for running those works, provided that you comply with 169 | the terms of this License in conveying all material for which you do 170 | not control copyright. Those thus making or running the covered works 171 | for you must do so exclusively on your behalf, under your direction 172 | and control, on terms that prohibit them from making any copies of 173 | your copyrighted material outside their relationship with you. 174 | 175 | Conveying under any other circumstances is permitted solely under 176 | the conditions stated below. Sublicensing is not allowed; section 10 177 | makes it unnecessary. 178 | 179 | 3. Protecting Users' Legal Rights From Anti-Circumvention Law. 180 | 181 | No covered work shall be deemed part of an effective technological 182 | measure under any applicable law fulfilling obligations under article 183 | 11 of the WIPO copyright treaty adopted on 20 December 1996, or 184 | similar laws prohibiting or restricting circumvention of such 185 | measures. 186 | 187 | When you convey a covered work, you waive any legal power to forbid 188 | circumvention of technological measures to the extent such circumvention 189 | is effected by exercising rights under this License with respect to 190 | the covered work, and you disclaim any intention to limit operation or 191 | modification of the work as a means of enforcing, against the work's 192 | users, your or third parties' legal rights to forbid circumvention of 193 | technological measures. 194 | 195 | 4. Conveying Verbatim Copies. 196 | 197 | You may convey verbatim copies of the Program's source code as you 198 | receive it, in any medium, provided that you conspicuously and 199 | appropriately publish on each copy an appropriate copyright notice; 200 | keep intact all notices stating that this License and any 201 | non-permissive terms added in accord with section 7 apply to the code; 202 | keep intact all notices of the absence of any warranty; and give all 203 | recipients a copy of this License along with the Program. 204 | 205 | You may charge any price or no price for each copy that you convey, 206 | and you may offer support or warranty protection for a fee. 207 | 208 | 5. Conveying Modified Source Versions. 209 | 210 | You may convey a work based on the Program, or the modifications to 211 | produce it from the Program, in the form of source code under the 212 | terms of section 4, provided that you also meet all of these conditions: 213 | 214 | a) The work must carry prominent notices stating that you modified 215 | it, and giving a relevant date. 216 | 217 | b) The work must carry prominent notices stating that it is 218 | released under this License and any conditions added under section 219 | 7. This requirement modifies the requirement in section 4 to 220 | "keep intact all notices". 221 | 222 | c) You must license the entire work, as a whole, under this 223 | License to anyone who comes into possession of a copy. This 224 | License will therefore apply, along with any applicable section 7 225 | additional terms, to the whole of the work, and all its parts, 226 | regardless of how they are packaged. This License gives no 227 | permission to license the work in any other way, but it does not 228 | invalidate such permission if you have separately received it. 229 | 230 | d) If the work has interactive user interfaces, each must display 231 | Appropriate Legal Notices; however, if the Program has interactive 232 | interfaces that do not display Appropriate Legal Notices, your 233 | work need not make them do so. 234 | 235 | A compilation of a covered work with other separate and independent 236 | works, which are not by their nature extensions of the covered work, 237 | and which are not combined with it such as to form a larger program, 238 | in or on a volume of a storage or distribution medium, is called an 239 | "aggregate" if the compilation and its resulting copyright are not 240 | used to limit the access or legal rights of the compilation's users 241 | beyond what the individual works permit. Inclusion of a covered work 242 | in an aggregate does not cause this License to apply to the other 243 | parts of the aggregate. 244 | 245 | 6. Conveying Non-Source Forms. 246 | 247 | You may convey a covered work in object code form under the terms 248 | of sections 4 and 5, provided that you also convey the 249 | machine-readable Corresponding Source under the terms of this License, 250 | in one of these ways: 251 | 252 | a) Convey the object code in, or embodied in, a physical product 253 | (including a physical distribution medium), accompanied by the 254 | Corresponding Source fixed on a durable physical medium 255 | customarily used for software interchange. 256 | 257 | b) Convey the object code in, or embodied in, a physical product 258 | (including a physical distribution medium), accompanied by a 259 | written offer, valid for at least three years and valid for as 260 | long as you offer spare parts or customer support for that product 261 | model, to give anyone who possesses the object code either (1) a 262 | copy of the Corresponding Source for all the software in the 263 | product that is covered by this License, on a durable physical 264 | medium customarily used for software interchange, for a price no 265 | more than your reasonable cost of physically performing this 266 | conveying of source, or (2) access to copy the 267 | Corresponding Source from a network server at no charge. 268 | 269 | c) Convey individual copies of the object code with a copy of the 270 | written offer to provide the Corresponding Source. This 271 | alternative is allowed only occasionally and noncommercially, and 272 | only if you received the object code with such an offer, in accord 273 | with subsection 6b. 274 | 275 | d) Convey the object code by offering access from a designated 276 | place (gratis or for a charge), and offer equivalent access to the 277 | Corresponding Source in the same way through the same place at no 278 | further charge. You need not require recipients to copy the 279 | Corresponding Source along with the object code. If the place to 280 | copy the object code is a network server, the Corresponding Source 281 | may be on a different server (operated by you or a third party) 282 | that supports equivalent copying facilities, provided you maintain 283 | clear directions next to the object code saying where to find the 284 | Corresponding Source. Regardless of what server hosts the 285 | Corresponding Source, you remain obligated to ensure that it is 286 | available for as long as needed to satisfy these requirements. 287 | 288 | e) Convey the object code using peer-to-peer transmission, provided 289 | you inform other peers where the object code and Corresponding 290 | Source of the work are being offered to the general public at no 291 | charge under subsection 6d. 292 | 293 | A separable portion of the object code, whose source code is excluded 294 | from the Corresponding Source as a System Library, need not be 295 | included in conveying the object code work. 296 | 297 | A "User Product" is either (1) a "consumer product", which means any 298 | tangible personal property which is normally used for personal, family, 299 | or household purposes, or (2) anything designed or sold for incorporation 300 | into a dwelling. In determining whether a product is a consumer product, 301 | doubtful cases shall be resolved in favor of coverage. For a particular 302 | product received by a particular user, "normally used" refers to a 303 | typical or common use of that class of product, regardless of the status 304 | of the particular user or of the way in which the particular user 305 | actually uses, or expects or is expected to use, the product. A product 306 | is a consumer product regardless of whether the product has substantial 307 | commercial, industrial or non-consumer uses, unless such uses represent 308 | the only significant mode of use of the product. 309 | 310 | "Installation Information" for a User Product means any methods, 311 | procedures, authorization keys, or other information required to install 312 | and execute modified versions of a covered work in that User Product from 313 | a modified version of its Corresponding Source. The information must 314 | suffice to ensure that the continued functioning of the modified object 315 | code is in no case prevented or interfered with solely because 316 | modification has been made. 317 | 318 | If you convey an object code work under this section in, or with, or 319 | specifically for use in, a User Product, and the conveying occurs as 320 | part of a transaction in which the right of possession and use of the 321 | User Product is transferred to the recipient in perpetuity or for a 322 | fixed term (regardless of how the transaction is characterized), the 323 | Corresponding Source conveyed under this section must be accompanied 324 | by the Installation Information. But this requirement does not apply 325 | if neither you nor any third party retains the ability to install 326 | modified object code on the User Product (for example, the work has 327 | been installed in ROM). 328 | 329 | The requirement to provide Installation Information does not include a 330 | requirement to continue to provide support service, warranty, or updates 331 | for a work that has been modified or installed by the recipient, or for 332 | the User Product in which it has been modified or installed. Access to a 333 | network may be denied when the modification itself materially and 334 | adversely affects the operation of the network or violates the rules and 335 | protocols for communication across the network. 336 | 337 | Corresponding Source conveyed, and Installation Information provided, 338 | in accord with this section must be in a format that is publicly 339 | documented (and with an implementation available to the public in 340 | source code form), and must require no special password or key for 341 | unpacking, reading or copying. 342 | 343 | 7. Additional Terms. 344 | 345 | "Additional permissions" are terms that supplement the terms of this 346 | License by making exceptions from one or more of its conditions. 347 | Additional permissions that are applicable to the entire Program shall 348 | be treated as though they were included in this License, to the extent 349 | that they are valid under applicable law. If additional permissions 350 | apply only to part of the Program, that part may be used separately 351 | under those permissions, but the entire Program remains governed by 352 | this License without regard to the additional permissions. 353 | 354 | When you convey a copy of a covered work, you may at your option 355 | remove any additional permissions from that copy, or from any part of 356 | it. (Additional permissions may be written to require their own 357 | removal in certain cases when you modify the work.) You may place 358 | additional permissions on material, added by you to a covered work, 359 | for which you have or can give appropriate copyright permission. 360 | 361 | Notwithstanding any other provision of this License, for material you 362 | add to a covered work, you may (if authorized by the copyright holders of 363 | that material) supplement the terms of this License with terms: 364 | 365 | a) Disclaiming warranty or limiting liability differently from the 366 | terms of sections 15 and 16 of this License; or 367 | 368 | b) Requiring preservation of specified reasonable legal notices or 369 | author attributions in that material or in the Appropriate Legal 370 | Notices displayed by works containing it; or 371 | 372 | c) Prohibiting misrepresentation of the origin of that material, or 373 | requiring that modified versions of such material be marked in 374 | reasonable ways as different from the original version; or 375 | 376 | d) Limiting the use for publicity purposes of names of licensors or 377 | authors of the material; or 378 | 379 | e) Declining to grant rights under trademark law for use of some 380 | trade names, trademarks, or service marks; or 381 | 382 | f) Requiring indemnification of licensors and authors of that 383 | material by anyone who conveys the material (or modified versions of 384 | it) with contractual assumptions of liability to the recipient, for 385 | any liability that these contractual assumptions directly impose on 386 | those licensors and authors. 387 | 388 | All other non-permissive additional terms are considered "further 389 | restrictions" within the meaning of section 10. If the Program as you 390 | received it, or any part of it, contains a notice stating that it is 391 | governed by this License along with a term that is a further 392 | restriction, you may remove that term. If a license document contains 393 | a further restriction but permits relicensing or conveying under this 394 | License, you may add to a covered work material governed by the terms 395 | of that license document, provided that the further restriction does 396 | not survive such relicensing or conveying. 397 | 398 | If you add terms to a covered work in accord with this section, you 399 | must place, in the relevant source files, a statement of the 400 | additional terms that apply to those files, or a notice indicating 401 | where to find the applicable terms. 402 | 403 | Additional terms, permissive or non-permissive, may be stated in the 404 | form of a separately written license, or stated as exceptions; 405 | the above requirements apply either way. 406 | 407 | 8. Termination. 408 | 409 | You may not propagate or modify a covered work except as expressly 410 | provided under this License. Any attempt otherwise to propagate or 411 | modify it is void, and will automatically terminate your rights under 412 | this License (including any patent licenses granted under the third 413 | paragraph of section 11). 414 | 415 | However, if you cease all violation of this License, then your 416 | license from a particular copyright holder is reinstated (a) 417 | provisionally, unless and until the copyright holder explicitly and 418 | finally terminates your license, and (b) permanently, if the copyright 419 | holder fails to notify you of the violation by some reasonable means 420 | prior to 60 days after the cessation. 421 | 422 | Moreover, your license from a particular copyright holder is 423 | reinstated permanently if the copyright holder notifies you of the 424 | violation by some reasonable means, this is the first time you have 425 | received notice of violation of this License (for any work) from that 426 | copyright holder, and you cure the violation prior to 30 days after 427 | your receipt of the notice. 428 | 429 | Termination of your rights under this section does not terminate the 430 | licenses of parties who have received copies or rights from you under 431 | this License. If your rights have been terminated and not permanently 432 | reinstated, you do not qualify to receive new licenses for the same 433 | material under section 10. 434 | 435 | 9. Acceptance Not Required for Having Copies. 436 | 437 | You are not required to accept this License in order to receive or 438 | run a copy of the Program. Ancillary propagation of a covered work 439 | occurring solely as a consequence of using peer-to-peer transmission 440 | to receive a copy likewise does not require acceptance. However, 441 | nothing other than this License grants you permission to propagate or 442 | modify any covered work. These actions infringe copyright if you do 443 | not accept this License. Therefore, by modifying or propagating a 444 | covered work, you indicate your acceptance of this License to do so. 445 | 446 | 10. Automatic Licensing of Downstream Recipients. 447 | 448 | Each time you convey a covered work, the recipient automatically 449 | receives a license from the original licensors, to run, modify and 450 | propagate that work, subject to this License. You are not responsible 451 | for enforcing compliance by third parties with this License. 452 | 453 | An "entity transaction" is a transaction transferring control of an 454 | organization, or substantially all assets of one, or subdividing an 455 | organization, or merging organizations. If propagation of a covered 456 | work results from an entity transaction, each party to that 457 | transaction who receives a copy of the work also receives whatever 458 | licenses to the work the party's predecessor in interest had or could 459 | give under the previous paragraph, plus a right to possession of the 460 | Corresponding Source of the work from the predecessor in interest, if 461 | the predecessor has it or can get it with reasonable efforts. 462 | 463 | You may not impose any further restrictions on the exercise of the 464 | rights granted or affirmed under this License. For example, you may 465 | not impose a license fee, royalty, or other charge for exercise of 466 | rights granted under this License, and you may not initiate litigation 467 | (including a cross-claim or counterclaim in a lawsuit) alleging that 468 | any patent claim is infringed by making, using, selling, offering for 469 | sale, or importing the Program or any portion of it. 470 | 471 | 11. Patents. 472 | 473 | A "contributor" is a copyright holder who authorizes use under this 474 | License of the Program or a work on which the Program is based. The 475 | work thus licensed is called the contributor's "contributor version". 476 | 477 | A contributor's "essential patent claims" are all patent claims 478 | owned or controlled by the contributor, whether already acquired or 479 | hereafter acquired, that would be infringed by some manner, permitted 480 | by this License, of making, using, or selling its contributor version, 481 | but do not include claims that would be infringed only as a 482 | consequence of further modification of the contributor version. For 483 | purposes of this definition, "control" includes the right to grant 484 | patent sublicenses in a manner consistent with the requirements of 485 | this License. 486 | 487 | Each contributor grants you a non-exclusive, worldwide, royalty-free 488 | patent license under the contributor's essential patent claims, to 489 | make, use, sell, offer for sale, import and otherwise run, modify and 490 | propagate the contents of its contributor version. 491 | 492 | In the following three paragraphs, a "patent license" is any express 493 | agreement or commitment, however denominated, not to enforce a patent 494 | (such as an express permission to practice a patent or covenant not to 495 | sue for patent infringement). To "grant" such a patent license to a 496 | party means to make such an agreement or commitment not to enforce a 497 | patent against the party. 498 | 499 | If you convey a covered work, knowingly relying on a patent license, 500 | and the Corresponding Source of the work is not available for anyone 501 | to copy, free of charge and under the terms of this License, through a 502 | publicly available network server or other readily accessible means, 503 | then you must either (1) cause the Corresponding Source to be so 504 | available, or (2) arrange to deprive yourself of the benefit of the 505 | patent license for this particular work, or (3) arrange, in a manner 506 | consistent with the requirements of this License, to extend the patent 507 | license to downstream recipients. "Knowingly relying" means you have 508 | actual knowledge that, but for the patent license, your conveying the 509 | covered work in a country, or your recipient's use of the covered work 510 | in a country, would infringe one or more identifiable patents in that 511 | country that you have reason to believe are valid. 512 | 513 | If, pursuant to or in connection with a single transaction or 514 | arrangement, you convey, or propagate by procuring conveyance of, a 515 | covered work, and grant a patent license to some of the parties 516 | receiving the covered work authorizing them to use, propagate, modify 517 | or convey a specific copy of the covered work, then the patent license 518 | you grant is automatically extended to all recipients of the covered 519 | work and works based on it. 520 | 521 | A patent license is "discriminatory" if it does not include within 522 | the scope of its coverage, prohibits the exercise of, or is 523 | conditioned on the non-exercise of one or more of the rights that are 524 | specifically granted under this License. You may not convey a covered 525 | work if you are a party to an arrangement with a third party that is 526 | in the business of distributing software, under which you make payment 527 | to the third party based on the extent of your activity of conveying 528 | the work, and under which the third party grants, to any of the 529 | parties who would receive the covered work from you, a discriminatory 530 | patent license (a) in connection with copies of the covered work 531 | conveyed by you (or copies made from those copies), or (b) primarily 532 | for and in connection with specific products or compilations that 533 | contain the covered work, unless you entered into that arrangement, 534 | or that patent license was granted, prior to 28 March 2007. 535 | 536 | Nothing in this License shall be construed as excluding or limiting 537 | any implied license or other defenses to infringement that may 538 | otherwise be available to you under applicable patent law. 539 | 540 | 12. No Surrender of Others' Freedom. 541 | 542 | If conditions are imposed on you (whether by court order, agreement or 543 | otherwise) that contradict the conditions of this License, they do not 544 | excuse you from the conditions of this License. If you cannot convey a 545 | covered work so as to satisfy simultaneously your obligations under this 546 | License and any other pertinent obligations, then as a consequence you may 547 | not convey it at all. For example, if you agree to terms that obligate you 548 | to collect a royalty for further conveying from those to whom you convey 549 | the Program, the only way you could satisfy both those terms and this 550 | License would be to refrain entirely from conveying the Program. 551 | 552 | 13. Use with the GNU Affero General Public License. 553 | 554 | Notwithstanding any other provision of this License, you have 555 | permission to link or combine any covered work with a work licensed 556 | under version 3 of the GNU Affero General Public License into a single 557 | combined work, and to convey the resulting work. The terms of this 558 | License will continue to apply to the part which is the covered work, 559 | but the special requirements of the GNU Affero General Public License, 560 | section 13, concerning interaction through a network will apply to the 561 | combination as such. 562 | 563 | 14. Revised Versions of this License. 564 | 565 | The Free Software Foundation may publish revised and/or new versions of 566 | the GNU General Public License from time to time. Such new versions will 567 | be similar in spirit to the present version, but may differ in detail to 568 | address new problems or concerns. 569 | 570 | Each version is given a distinguishing version number. If the 571 | Program specifies that a certain numbered version of the GNU General 572 | Public License "or any later version" applies to it, you have the 573 | option of following the terms and conditions either of that numbered 574 | version or of any later version published by the Free Software 575 | Foundation. If the Program does not specify a version number of the 576 | GNU General Public License, you may choose any version ever published 577 | by the Free Software Foundation. 578 | 579 | If the Program specifies that a proxy can decide which future 580 | versions of the GNU General Public License can be used, that proxy's 581 | public statement of acceptance of a version permanently authorizes you 582 | to choose that version for the Program. 583 | 584 | Later license versions may give you additional or different 585 | permissions. However, no additional obligations are imposed on any 586 | author or copyright holder as a result of your choosing to follow a 587 | later version. 588 | 589 | 15. Disclaimer of Warranty. 590 | 591 | THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY 592 | APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT 593 | HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY 594 | OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, 595 | THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 596 | PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM 597 | IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF 598 | ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 599 | 600 | 16. Limitation of Liability. 601 | 602 | IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 603 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS 604 | THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY 605 | GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE 606 | USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF 607 | DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD 608 | PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), 609 | EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF 610 | SUCH DAMAGES. 611 | 612 | 17. Interpretation of Sections 15 and 16. 613 | 614 | If the disclaimer of warranty and limitation of liability provided 615 | above cannot be given local legal effect according to their terms, 616 | reviewing courts shall apply local law that most closely approximates 617 | an absolute waiver of all civil liability in connection with the 618 | Program, unless a warranty or assumption of liability accompanies a 619 | copy of the Program in return for a fee. 620 | 621 | END OF TERMS AND CONDITIONS 622 | 623 | How to Apply These Terms to Your New Programs 624 | 625 | If you develop a new program, and you want it to be of the greatest 626 | possible use to the public, the best way to achieve this is to make it 627 | free software which everyone can redistribute and change under these terms. 628 | 629 | To do so, attach the following notices to the program. It is safest 630 | to attach them to the start of each source file to most effectively 631 | state the exclusion of warranty; and each file should have at least 632 | the "copyright" line and a pointer to where the full notice is found. 633 | 634 | 635 | Copyright (C) 636 | 637 | This program is free software: you can redistribute it and/or modify 638 | it under the terms of the GNU General Public License as published by 639 | the Free Software Foundation, either version 3 of the License, or 640 | (at your option) any later version. 641 | 642 | This program is distributed in the hope that it will be useful, 643 | but WITHOUT ANY WARRANTY; without even the implied warranty of 644 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 645 | GNU General Public License for more details. 646 | 647 | You should have received a copy of the GNU General Public License 648 | along with this program. If not, see . 649 | 650 | Also add information on how to contact you by electronic and paper mail. 651 | 652 | If the program does terminal interaction, make it output a short 653 | notice like this when it starts in an interactive mode: 654 | 655 | Copyright (C) 656 | This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 657 | This is free software, and you are welcome to redistribute it 658 | under certain conditions; type `show c' for details. 659 | 660 | The hypothetical commands `show w' and `show c' should show the appropriate 661 | parts of the General Public License. Of course, your program's commands 662 | might be different; for a GUI interface, you would use an "about box". 663 | 664 | You should also get your employer (if you work as a programmer) or school, 665 | if any, to sign a "copyright disclaimer" for the program, if necessary. 666 | For more information on this, and how to apply and follow the GNU GPL, see 667 | . 668 | 669 | The GNU General Public License does not permit incorporating your program 670 | into proprietary programs. If your program is a subroutine library, you 671 | may consider it more useful to permit linking proprietary applications with 672 | the library. If this is what you want to do, use the GNU Lesser General 673 | Public License instead of this License. But first, please read 674 | . 675 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # APRSDB Overview 2 | #### by Bill Mitchell, AE0EE 3 | 4 | ## Introduction 5 | APRSDB is a set of programs and code that work together to provide a way to store APRS data. Analysis of this data can provide insight on radio propagation, network traffic, and identify potential problems in digipeater configuration. 6 | 7 | This project was inspired by NG0E's VHF Propagation website, which uses APRS-IS data. Unlike that system, APRSDB uses only local data, and is designed to be fully independent of the internet. Furthermore, it analyzes all the packets it reeceives, where APRS-IS seems to remove duplicates---throwing out perfectly good data. 8 | 9 | At this time APRSDB takes some work to set up and is only operational on Linux. I am not prepared to support other operating systems, but all the tools are cross-platform so it should be feasible. 10 | 11 | This platform is split into two major parts: the collection/database back-end, and the yet-to-be-released web front-end (_aprs-moose_). These two parts are split because it should make keeping things updated easier. For now, most GIS software should be able to interact with the data; _QGIS_ certainly can. Common queries have been saved as views for easy access. 12 | 13 | ## How It Works 14 | There are several steps involved in the storage, analysis, and display of APRS data. Different open-source programs are used for these steps. 15 | 16 | * aprsdb (back-end) 17 | ** _Direwolf_ (soundcard packet decoder) 18 | ** _aprs-python_ (Python3-based APRS parser) 19 | ** _aprsdb_ (Python3 code for database insertion) 20 | ** _Postgres_ (database) with _PostGIS_ extensions 21 | * aprs-moose (front-end) 22 | ** _Apache_ (webserver) 23 | ** _MapServer_ (mapping backend) 24 | ** _GeoMoose_ (javascript map frontend) 25 | 26 | If you are interested in more detailed spatial analysis, use a full GIS program, e.g. _QGIS_ 27 | 28 | Additional scripts, configuration files, and mapfiles provide a base from which to work without requiring expert knowledge of each of the programs in the stack. 29 | 30 | ## Installation 31 | The above programs need to be installed. On Debian-based systems try 32 | `sudo apt-get install postgres postgis qgis apache2 git` 33 | 34 | Get Direwolf and the Python dependencies 35 | ```bash 36 | mkdir ~/Install 37 | cd ~/Install 38 | git clone https://github.com/wsmitchell3/aprs-python 39 | git clone https://github.com/wsmitchell3/aprsdb 40 | git clone https://github.com/wb2osz/direwolf 41 | sudo pip3 install ~/Install/aprs-python 42 | sudo pip3 install psycopg2 43 | sudo pip3 install gpsd-py3 44 | ``` 45 | 46 | Install Direwolf 47 | ```bash 48 | cd ~/Install/direwolf 49 | make 50 | sudo make install 51 | make install-conf 52 | cd ~ 53 | ``` 54 | You will need to edit the installed direwolf configuration to add your callsign, etc. 55 | 56 | Enable PostGIS (as a superuser), then create the users _moose_ and _radio_. We will later restrict permissions for these users. If you want to customize the passwords for these accounts, edit the db_roles.sql file before running it. 57 | ```bash 58 | $ sudo su postgres 59 | # createdb aprs 60 | # psql -d aprs -f /home/[user]/Install/aprsdb/enable_postgis.sql 61 | # psql -d aprs -f /home/[user]/Install/aprsdb/db_roles.sql 62 | # exit 63 | ``` 64 | 65 | Create the database tables, views, and indexes. The default password for the radio user is found in db_roles.sql 66 | ```bash 67 | $ psql -d aprs -h 127.0.0.1 -p 5432 --user radio -f ~/Install/aprsd/aprsdb_creation.sql 68 | ``` 69 | 70 | Update (restrict) the _moose_ access privileges (read-only) and grant the _radio_ access privileges (write, but not DB admin) 71 | *NOTE!* This script is intended for use on a single-database system, and may well have adverse side-effects on other databases. Use with care. 72 | ```bash 73 | $ sudo su postgres 74 | postgres$ psql -d aprs -f /home/[user]/Install/aprsdb/db_access_privs.sql 75 | postgres$ exit 76 | ``` 77 | 78 | Copy *generic_aprsdb.conf* to _~/aprsdb.conf_, and update _~/aprsdb.conf_ with the appropriate _Receive station info_ and any configuration customizations you made. If you don't want the config file in your home directory, it can be specified via the command line --config (-c) option. 79 | 80 | ## Usage 81 | ### Real-time on-air 82 | Provide an audio input for direwolf. This can either be through an SDR or by hooking up a radio's output to the line-in of your soundcard. You will likely need to set this in the _direwolf.conf_ file. 83 | 84 | Start direwolf. I usually prefer to do this using the _screen_ command, in a way that invokes direwolf then immediately detaches: 85 | ```bash 86 | $ screen -S direwolf -d -m direwolf 87 | ``` 88 | You can get back to the screen with 89 | ```bash 90 | $ screen -r direwolf 91 | ``` 92 | 93 | Once direwolf is running, we can use the _kissutil_ program to give us the plaintext. This can be sent to a file for later processing, sent straight into the database, or both. 94 | ```bash 95 | $ # Option 2: direct to database 96 | $ screen -S kissutil 97 | $ kissutil | python ~/Install/aprsdb/aprsdb.py 98 | ``` 99 | 100 | ### Off-line 101 | Text output from kissutil or other raw APRS text can be fed directly into the database: 102 | ```bash 103 | $ cat [input] | python ~/Install/aprsdb/aprsdb.py 104 | ``` 105 | -------------------------------------------------------------------------------- /aprsdb.py: -------------------------------------------------------------------------------- 1 | #! /usr/bin/python 2 | 3 | # 20180315_aprs-db.py 4 | # by Bill Mitchell 5 | 6 | # Functions for moving APRS data from aprslib into a psql database 7 | 8 | import psycopg2 # Database interface 9 | import aprslib # APRS parsing 10 | import sys, os # General system utilities 11 | import argparse # Parse arguments 12 | import datetime, time, re, configparser, getpass # Time, regex, config, and password entry 13 | import decimal # For truncating floats (e.g. lat/long) 14 | from psycopg2 import sql 15 | try: 16 | import gpsd # Use the GPS library if we have it 17 | import aprsgps # custom functions that require gpsd 18 | use_gps = True 19 | except: 20 | use_gps = False # GPS library not found (nor required) 21 | 22 | # Parse arguments 23 | parser = argparse.ArgumentParser() 24 | parser.add_argument('-c', '--config', help='APRSDB config file') 25 | args = parser.parse_args(sys.argv[1:]) 26 | # Read the config file 27 | config = configparser.ConfigParser() 28 | try: 29 | config.read_file(open(vars(args)['config'])) # Try to open the preferred config 30 | except TypeError: 31 | try: 32 | config.read_file(open(os.path.expanduser("~") + os.path.sep + 'aprsdb.conf')) # Default to a locally-configured config 33 | except FileNotFoundError: 34 | try: 35 | config.read_file(open(os.path.split(sys.argv[0])[0] + 'aprsdb.conf')) # Fallback to the install directory 36 | except FileNotFoundError: 37 | config.read_file(open(os.path.join(os.path.split(sys.argv[0])[0], 'generic_aprsdb.conf'))) # Fallback to the generic one 38 | 39 | # Set info for receive station 40 | rxinfo={'call':config.get('aprs', 'rxcall'), 'symbol':config.get('aprs', 'rxsymbol'), 'symbol_table': config.get('aprs', 'rxtable'), 'latitude':config.get('aprs', 'latitude'), 'longitude':config.get('aprs','longitude')} 41 | 42 | 43 | # Set up the timestamp regex 44 | dwts = re.compile("^\[([0-9]*?) ([0-9]{4})([0-9]{2})([0-9]{2})_([0-9]{2})([0-9]{2})([0-9]{2})\]") # Match direwolf timestamp format [0 YYYYMMDD_hhmmss], grouped conveniently 45 | 46 | 47 | # Set up data for database connection 48 | session_id = 0 # Will attempt to update 49 | my_schema = {'common':[], 'aprsdb_errs':[], 'location':[], 'map_entry':[], 'mic_e':[], 'thirdparty':[], 'uncompressed':[], 'compressed':[], 'status':[], 'object':[], 'wx':[], 'message':[], 'telemetry_message':[]} # Fields will be drawn from the database itself 50 | 51 | conn=None 52 | try: # Establish database connection 53 | if config.getboolean('psqlw', 'localuser')==True: # Use local user authentication 54 | conn=psycopg2.connect(dbname=config.get('psql', 'dbname'), user=getpass.getuser()) 55 | else: # Use authentication from config 56 | conn = psycopg2.connect(dbname=config.get('psql', 'dbname'), user=config.get('psqlw', 'dbuser'), host=config.get('psqlw', 'dbhost'), port=config.get('psqlw','dbport'), password=config.get('psqlw', 'dbpass')) 57 | except: 58 | print("Unable to connect to the database") 59 | raise 60 | cur = conn.cursor() # Create a database cursor 61 | 62 | try: # Establish session ID and time 63 | cur.execute("INSERT INTO sessions (start_time_utc_s, session_offset) VALUES (%s, %s) RETURNING session_id;", (time.time(), 0)) 64 | session_id = cur.fetchone()[0] 65 | conn.commit() 66 | except: 67 | print("Error entering session metadata to database") 68 | raise # DB connection is mission-critical, so error out here 69 | 70 | for packet_format in my_schema: # Get the column names for each packet format 71 | cur.execute("SELECT column_name FROM information_schema.columns WHERE table_name=%s;", (packet_format,)) # Query column names for the table 72 | my_schema[packet_format] = [x[0] for x in cur.fetchall()] # Update column names 73 | 74 | def check_rx_station(conn, parsed): 75 | """ 76 | Check if the receiving station is in digis and location, adding if needed. 77 | conn: psycopg2 database connection 78 | parsed: dictionary with fields call, symbol, symbol_table, latitude, and longitude (e.g. rxinfo) 79 | returns: void 80 | """ 81 | cur = conn.cursor() 82 | mylid = None 83 | [digi_id, mycall, mysym, mytable, oldloc] = [None for _ in range(5)] 84 | try: 85 | # Look to see if rx "digi" is known 86 | cur.execute("SELECT * FROM digis WHERE call=%s;", (parsed['call'],)) 87 | myresult = cur.fetchone() 88 | # Get the info for a known digi 89 | if myresult != None: 90 | [digi_id, mycall, mysym, mytable, oldloc] = myresult 91 | 92 | # Compute the linestring for the rx location 93 | cur.execute("SELECT ST_SetSRID(ST_MakePoint(CAST(%s AS FLOAT), CAST(%s AS FLOAT)), 4326);", (parsed['longitude'],parsed['latitude'])) 94 | myloc = cur.fetchone()[0] 95 | 96 | # If the rx digi is new, add it 97 | if digi_id == None: 98 | cur.execute("INSERT INTO digis (call, aprs_sym, aprs_table, loc) VALUES (%s, %s, %s, %s)", (parsed['call'], parsed['symbol'], parsed['symbol_table'], myloc)) 99 | 100 | else: # Digi is known 101 | # Check if rx digi has changed info 102 | if not (myloc==oldloc and mysym == parsed['symbol'] and mytable == parsed['symbol_table']): 103 | # Update rx digi info if needed 104 | cur.execute("UPDATE digis SET aprs_sym=%s, aprs_table=%s, loc=%s WHERE digi_id=%s;", (parsed['symbol'], parsed['symbol_table'], myloc, digi_id)) 105 | 106 | # Check for an existing entry in the location table 107 | cur.execute("SELECT lid FROM location WHERE linestring=%s;", (myloc,)) 108 | mylid = cur.fetchone() 109 | if mylid == None: # If location is new, add it 110 | cur.execute("INSERT INTO location (latitude, longitude, linestring) VALUES(%s, %s, %s) RETURNING lid;", (parsed['latitude'], parsed['longitude'], myloc)) 111 | mylid = cur.fetchone() 112 | mylid = mylid[0] 113 | 114 | conn.commit() 115 | except: 116 | conn.rollback() # If errors are encountered, abort 117 | raise 118 | return mylid 119 | 120 | 121 | def insert_sql_from_dict(table, mydict, codastring=''): 122 | """ 123 | Create an SQL query for inserting the keys/values in mydict into table 124 | table: the table where values will be inserted 125 | mydict: dictionary of field:value pairs, all of which will be inserted 126 | codastring: string or sql.SQL to go at the query's end ('RETURNING pid') 127 | returns: psycopg2.sql.Composed query object for use with a cursor 128 | """ 129 | myquery = sql.SQL("INSERT INTO {} ({}) VALUES ({}) {};").format( 130 | sql.Identifier(table), 131 | sql.SQL(', ').join(map(sql.Identifier, mydict.keys())), 132 | sql.SQL(', ').join(map(sql.Placeholder, mydict.keys())), 133 | sql.SQL(codastring)) 134 | 135 | return(myquery) 136 | 137 | def insert_digi(parsed, cur): 138 | """Insert values for a new digipeater. 139 | parsed: location packet for digi to be inserted 140 | cur: database cursor 141 | """ 142 | # Check that required information is present 143 | for key in ['longitude','latitude','src','symbol','symbol_table']: 144 | if (key not in parsed.keys()): 145 | raise KeyError("Missing key: " + key) # Something went badly wrong 146 | # Data checked basic test, insert it. 147 | try: 148 | cur.execute("INSERT INTO digis (call, aprs_sym, aprs_table, loc) VALUES (%s, %s, %s, ST_SetSRID(ST_MakePoint(CAST(%s AS FLOAT), CAST(%s AS FLOAT)), 4326));", (parsed['src'], parsed['symbol'], parsed['symbol_table'], parsed['longitude'], parsed['latitude'])) 149 | except: 150 | raise 151 | 152 | def check_update_digi(parsed, digi_id, cur): 153 | """ 154 | Check if digipeater location has changed, updating if necessary 155 | parsed: aprslib parsed packet for digi to be checked/updated 156 | digi_id: digi_id from digis table to be checked against, and updated if needed 157 | cur: psycopg2 database cursor 158 | """ 159 | # Check for requisite info, error if missing 160 | for key in ['longitude','latitude','src','symbol','symbol_table']: 161 | if (key not in parsed.keys()): 162 | raise KeyError("Missing key: " + key) 163 | 164 | # Get the old symbol for this digi 165 | cur.execute("SELECT aprs_sym FROM digis WHERE call=%s;", (parsed['src'],)) 166 | old_sym = cur.fetchone()[0] 167 | 168 | if (old_sym == '#' and parsed['symbol'] != old_sym): # Ignore non-unique gates 169 | return 170 | 171 | # Get existing digi location, symbol, and symbol table 172 | cur.execute("SELECT loc, aprs_sym, aprs_table FROM digis WHERE digi_id=%s;", (digi_id,)) 173 | [old_loc, old_sym, old_table] = cur.fetchone() 174 | 175 | # Prepare the new linestring with the digi's location 176 | if 'linestring' not in parsed.keys(): 177 | cur.execute("SELECT ST_SetSRID(ST_MakePoint(CAST(%s AS FLOAT), CAST(%s AS FLOAT)), 4326);", (parsed['longitude'], parsed['latitude'])) 178 | curr_loc = cur.fetchone()[0] 179 | else: 180 | curr_loc = parsed['linestring'] 181 | 182 | # Update digi record if it has moved or changed symbol [table] 183 | if ((curr_loc != old_loc) or (parsed['symbol'] != old_sym) or (parsed['symbol_table'] != old_table)): 184 | cur.execute("UPDATE digis SET loc=%s, aprs_sym=%s, aprs_table=%s WHERE digi_id=%s;", (curr_loc, parsed['symbol'], parsed['symbol_table'], digi_id)) 185 | 186 | 187 | 188 | def process_digi(parsed, cur): 189 | """Insert values for a new digipeater from a parsed packet 190 | parsed: an aprslib parsed packet 191 | cur: psycopg2 database cursor 192 | """ 193 | 194 | # Check for crucial information 195 | if ('symbol' not in parsed.keys()): 196 | raise KeyError("Missing key: symbol") 197 | 198 | # Get existing record for digi 199 | cur.execute("SELECT digi_id, aprs_sym FROM digis WHERE call=%s;", (parsed['src'],)) 200 | if (cur.rowcount == 0): # No matching digi 201 | insert_digi(parsed, cur) 202 | else: #Digis are forced unique by callsign, so can only be one row matching 203 | myresult = cur.fetchone() 204 | # Check if updates are needed and make them 205 | check_update_digi(parsed, myresult[0], cur) 206 | 207 | 208 | def remove_NULL_path(path): 209 | """ 210 | Remove NULL entries from APRS path 211 | path: path of parsed packet (list of strings) 212 | returns: list of digipeaters with NULL entries removed, in order 213 | """ 214 | digipath = [] 215 | for digi in path: 216 | if digi!='NULL': 217 | digipath.append(digi) 218 | return(digipath) 219 | 220 | def process_path(path, packet_id, conn): 221 | """ 222 | Insert the path routing info to the database 223 | path: Python list of path routing elements (e.g. N0QVC-1, WIDE1*, N0PBA-1,WIDE2-1) 224 | packet_id: packet_id from packets table corresponding to the entry being processed 225 | conn: psycopg2 database connection 226 | """ 227 | cur = conn.cursor() 228 | path = aprslib.util.remove_WIDEn_N(path) # Get rid of WIDEn-N and asterisks 229 | if 'NULL' in path: 230 | path.remove('NULL') # remove first NULL value 231 | 232 | digi_src = False 233 | cur.execute("SELECT d1.call FROM common AS c1 INNER JOIN digis AS d1 ON d1.call=c1.src WHERE c1.pid=%s;", (packet_id, )) # Check if source is a known digi 234 | if cur.rowcount == 1: # 1 digi matched 235 | myresult = cur.fetchone()[0] 236 | path.insert(0, myresult) # Add first hop to inter-digi list 237 | digi_src = True 238 | 239 | 240 | if (path == None or path == []): # No route info 241 | return 242 | 243 | for call in path: # Check for new digis! 244 | cur.execute("SELECT call FROM digis WHERE call=%s;", (call,)) 245 | myresult = cur.fetchone() 246 | if myresult == None: # Found new digi 247 | cur.execute("INSERT INTO digis (call) VALUES (%s);", (call,)) 248 | 249 | path.append(rxinfo['call']) # All packets end at RX site 250 | 251 | for i in range(len(path)-1): # Split path into single hops 252 | src = path[i] 253 | dest = path[i+1] 254 | hop = i+1 255 | if (digi_src): # 0-index for digi-sourced packets 256 | hop = i 257 | route_id = 0 258 | 259 | # Check if route exists, and get its ID if it does 260 | cur.execute("SELECT route_id FROM routes WHERE src=%s AND dest=%s;", (src, dest)) 261 | myresult = cur.fetchone() 262 | if myresult != None: # Route exists 263 | route_id = myresult[0] 264 | else: # Route is new 265 | cur.execute("INSERT INTO routes (src, dest) VALUES (%s, %s) RETURNING route_id;", (src, dest)) 266 | route_id = cur.fetchone()[0] 267 | 268 | # Add the hop and route to the paths table 269 | cur.execute("INSERT INTO paths (pid, hop, route_id) VALUES (%s, %s, %s);", (packet_id, hop, route_id)) 270 | 271 | conn.commit() 272 | 273 | 274 | def process_packet(packet, conn, rxtime=None, is_subpacket=False): 275 | """ 276 | Load an unparsed APRS packet into the database 277 | packet: APRS packet string 278 | conn: psycopg2 database connection 279 | rxtime: time packet was received, as seconds since epoch (1/1/1970); if omitted or wrong format, uses system clock 280 | is_subpacket: boolean flag for whether this is a sub-packet 281 | returns packet_id (positive bigint) if successful, negative integer if not 282 | """ 283 | # Check for reasonable timestamp 284 | if (type(rxtime) is not float and type(rxtime) is not int or (rxtime is None)): 285 | rxtime=time.time() 286 | 287 | try: # Parse it 288 | parsed = aprslib.parse(packet) 289 | except aprslib.exceptions.ParseError as pe: 290 | try: # Salvage what data we can from the header of an unparseable packet 291 | parsed = pe.parsed 292 | parsed['format']="parseerror" 293 | except: # Couldn't salvage anything 294 | print("Unable to partially parse packet: '" + packet + "' at time "+ str(rxtime)) # DEBUG 295 | return(-6) # Unable to partially parse packet 296 | 297 | except aprslib.exceptions.UnknownFormat as uf: 298 | # Save what headers we can if the format is unknown 299 | parsed = uf.parsed 300 | parsed['format']="unknown" 301 | except: 302 | # Something else went wrong 303 | print("Unable to parse packet") # DEBUG 304 | try: 305 | cur=conn.cursor() 306 | cursor.execute("INSERT INTO aprsdb_errs (rxtime, rxsession, raw, message) VALUES (%s, %s, %s, %s);", (rxtime, session_id, packet, "Unable to parse packet")) 307 | except: 308 | pass 309 | return(-5) # Unable to parse packet 310 | 311 | # Add the receiving station metadata to the parsed data 312 | parsed.update({'rxtime':rxtime, 'rxsession':session_id, 'is_subpacket':is_subpacket}) 313 | 314 | # Send the parsed data on for further processing 315 | return(process_parsed(parsed, conn, rxtime, is_subpacket)) 316 | 317 | def process_parsed(parsed, conn, rxtime=time.time(), is_subpacket=False): 318 | """ 319 | Load a parsed APRS packet into the database 320 | parsed: parsed aprs packet dictionary from aprslib 321 | conn: psycopg2 database connection 322 | rxtime: time packet was received, as seconds since epoch (1/1/1970); if omitted or wrong format, uses system clock 323 | is_subpacket: boolean flag for whether this is a sub-packet 324 | returns packet_id (positive bigint) if successful, negative integer if not 325 | """ 326 | parsed['is_subpacket']=is_subpacket 327 | parsed['rxtime']=rxtime 328 | parsed['rx_loc_id'] = rxinfo['rx_loc_id'] # Get current loc_id for rx station 329 | if use_gps: #GPS features enabled (i.e. rover) 330 | if time.time() - rxinfo['gps_loc_time'] > 30: # seconds since prev. GPS point 331 | try: 332 | (mylat, mylon) = aprsgps.getLoc2D() # Get truncated coordinates 333 | rxinfo['latitude'] = mylat 334 | rxinfo['longitude'] = mylon 335 | rxinfo['rx_loc_id'] = check_rx_station(conn, rxinfo) # Update rx_station location, getting the location id back (for inserting in the common table) 336 | parsed['rx_loc_id'] = rxinfo['rx_loc_id'] # Put it into the parsed packet structure for insertion 337 | rxinfo['gps_loc_time'] = time.time() # Update the timestamp for the rx_loc 338 | except: 339 | print("Unable to get GPS coordinates") # DEBUG 340 | 341 | 342 | # Work around potential SQL reserved words, characters, and case-sensitivity 343 | parsed['dest'] = parsed.pop('to') 344 | parsed['src'] = parsed.pop('from') 345 | if parsed['format'] == 'mic-e': 346 | parsed['format'] = 'mic_e' 347 | if parsed['format'] == 'telemetry-message': 348 | parsed['format'] = 'telemetry_message' 349 | try: 350 | parsed['addressee'] = parsed['addresse'] 351 | except: 352 | pass 353 | 354 | # Ensure all field names are lower-case 355 | parsed = dict((k.lower(), v) for k,v in parsed.items()) 356 | 357 | 358 | # Insert data into common table 359 | # Start by finding the fields we have that go in common 360 | in_common = {x: parsed[x] for x in my_schema['common'] if x in parsed} 361 | try: 362 | # Put them in, and get the pid back for future reference 363 | cur.execute(insert_sql_from_dict('common', in_common, 'RETURNING pid'), in_common) 364 | mypacketid = cur.fetchone()[0] 365 | parsed['pid']=mypacketid 366 | conn.commit() 367 | except psycopg2.DataError as de: 368 | conn.rollback() 369 | print(de.pgerror) 370 | parsed['msg']=de.pgerror 371 | try: 372 | in_errs = {x:parsed[x] for x in my_schema['aprsdb_errs'] if x in parsed} 373 | cur.execute(insert_sql_from_dict('aprsdb_errs', in_errs), in_errs) 374 | conn.commit() 375 | return -7 376 | except: 377 | conn.rollback() 378 | raise 379 | return -8 380 | except: 381 | # Take it all back if something fails 382 | conn.rollback() 383 | raise 384 | return -2 # Unable to insert common table data 385 | 386 | # Check for digis 387 | if 'symbol' in parsed.keys(): 388 | # Digis and igates are # and & 389 | if parsed['symbol'] in ['#','&']: 390 | process_digi(parsed, cur) 391 | # Watch out for digis not using standard symbols 392 | cur.execute("SELECT call FROM digis WHERE call=%s;", (parsed['src'],)) 393 | if cur.fetchone() != None: # Call is a known digi 394 | if parsed['format'] not in ('object','item'): # Don't use digipeater data from objects or items 395 | process_digi(parsed, cur) 396 | 397 | # Handle third-party packets 398 | if parsed['format']=='thirdparty': 399 | # Source is a digi; check that it is known 400 | cur.execute("SELECT call FROM digis WHERE call=%s;", (parsed['src'],)) 401 | myresult = cur.fetchone() 402 | if myresult == None: # Found new digi 403 | cur.execute("INSERT INTO digis (call) VALUES (%s);", (parsed['src'],)) 404 | 405 | # Get the subpacket type 406 | parsed['subpacket_type'] = parsed['subpacket']['format'] 407 | # Process the subpacket, getting its pid for back reference 408 | parsed['subpacket_id'] = process_parsed(parsed['subpacket'], conn, rxtime, True) 409 | if parsed['subpacket_id'] <0: # Error encountered 410 | conn.rollback() 411 | return -4 # Unable to handle third-party packet 412 | 413 | # Handle weather packets 414 | if 'weather' in parsed.keys(): 415 | parsed['has_wx']=True 416 | for key in parsed['weather'].keys(): 417 | parsed[key]=parsed['weather'][key] 418 | if parsed['format']!='wx': # Watch out for objects/positions with weather 419 | # Find wx fields in the packet and their values 420 | in_schema = {x: parsed[x] for x in my_schema['wx'] if x in parsed} 421 | for x in in_schema: 422 | if type(in_schema[x]) is dict: 423 | # Stringify dictionaries 424 | in_schema[x] = str(in_schema[x]) 425 | # wx format packets will be entered later, but objects need their wx data entered now 426 | cur.execute(insert_sql_from_dict('wx', in_schema), in_schema) 427 | conn.commit() 428 | 429 | # Handle linestring creation and location entries 430 | if 'latitude' in parsed.keys() and 'longitude' in parsed.keys(): 431 | try: 432 | # Get the linestring (text representation of geospatial data) 433 | cur.execute("SELECT ST_SetSRID(ST_MakePoint(CAST(%s AS FLOAT), CAST(%s AS FLOAT)), 4326);", (parsed['longitude'], parsed['latitude'])) 434 | parsed['linestring'] = cur.fetchone()[0] 435 | 436 | # Check if we know this location 437 | cur.execute("SELECT lid FROM location WHERE latitude=%s AND longitude=%s;", (parsed['latitude'], parsed['longitude'])) 438 | myresult = cur.fetchall() 439 | if len(myresult)==0: # No results found 440 | # Enter location data into a table 441 | in_schema = {x: parsed[x] for x in my_schema['location'] if x in parsed} 442 | for x in in_schema: 443 | # Stringify dictionaries 444 | if type(in_schema[x]) is dict: 445 | in_schema[x] = str(in_schema[x]) 446 | cur.execute(insert_sql_from_dict('location', in_schema, 'RETURNING lid'), in_schema) 447 | parsed['lid']=cur.fetchall()[0] 448 | conn.commit() 449 | else: # Location known, we just need its id 450 | parsed['lid']=myresult[0] 451 | 452 | # Enter symbol/table, course/speed, and PHG into map_entry table 453 | in_schema = {x: parsed[x] for x in my_schema['map_entry'] if x in parsed} 454 | for x in in_schema: 455 | if type(in_schema[x]) is dict: 456 | in_schema[x] = str(in_schema[x]) 457 | cur.execute(insert_sql_from_dict('map_entry', in_schema), in_schema) 458 | 459 | conn.commit() 460 | except: 461 | conn.rollback() 462 | raise 463 | 464 | 465 | # Insert format-specific data into proper table 466 | try: 467 | in_schema = {x: parsed[x] for x in my_schema[parsed['format']] if x in parsed} 468 | except KeyError: 469 | # Format missing; salvage path info 470 | conn.commit() 471 | # Process path, but only if the path is RF 472 | if is_subpacket==False: 473 | process_path(parsed['path'], mypacketid, conn) 474 | return mypacketid 475 | 476 | # Get the main packet data ready for insertion 477 | for x in in_schema: 478 | if type(in_schema[x]) is dict: # Stringify dictionaries 479 | in_schema[x] = str(in_schema[x]) 480 | try: 481 | # Insert the packet's data into the format table 482 | cur.execute(insert_sql_from_dict(parsed['format'], in_schema), in_schema) 483 | conn.commit() 484 | except: 485 | conn.rollback() 486 | raise 487 | #return -3 488 | 489 | # Process path, only for RF paths 490 | if is_subpacket==False: 491 | process_path(parsed['path'], mypacketid, conn) 492 | 493 | return mypacketid 494 | 495 | 496 | def set_session_offset (session_id, conn, start_year=2018, start_month=1, start_day=1, start_hour=0, start_minute=0, start_second=0, start_usec=0, start_tz=datetime.timezone.utc): 497 | """ 498 | Set the session_offest field for a session, given a starting date/time. 499 | conn: psycopg2 database cursor 500 | start_tz: defaults to UTC 501 | Returns: True if successful, False if error 502 | """ 503 | true_start = datetime.datetime(start_year, start_month, start_day, start_hour, start_minute, start_second, start_usec, tzinfo=start_tz) 504 | cur = conn.cursor() 505 | cur.execute("SELECT start_time_utc_s FROM sessions WHERE session_id=%s;", (session_id,)) 506 | apparent_start = datetime.datetime.fromtimestamp(cur.fetchone()[0], tz=datetime.timezone.utc) 507 | 508 | dt = (true_start - apparent_start).total_seconds() 509 | try: 510 | cur.execute("UPDATE sessions SET session_offset=%s WHERE session_id=%s;", (dt, session_id)) 511 | conn.commit() 512 | except: 513 | conn.rollback() 514 | return(False) 515 | return(True) 516 | 517 | def get_direwolf_timestamp(packet): 518 | """ 519 | Parse the timestamp from a direwolf packet header 520 | packet: the direwolf output line to decode 521 | returns: seconds since 1970 (possibly fractional) or None 522 | """ 523 | # This really needs to be redone to pull the format from the config 524 | # Use %z flag for UTC offset 525 | # Use %s flag for epoch 526 | myresult = re.match(dwts, packet) 527 | try: 528 | (Y, m, d, H, M, S) = (int(x) for x in myresult.group(2,3,4,5,6,7)) 529 | except: 530 | return None 531 | 532 | packet_time = datetime.datetime(Y, m, d, H, M, S).timestamp()-time.localtime().tm_gmtoff 533 | 534 | def process_direwolf(line): 535 | """Parse direwolf output, returning the packet and epoch timestamp (or None). 536 | line: single line of raw Direwolf output 537 | returns: (channel=None, epoch=None, packet) 538 | """ 539 | # Build regex for the direwolf header (radio channel and optional timestamp) 540 | # Save the packet, too 541 | m = re.match("^\[([0-9]*) *([0-9_\-]*)] (.*)", line) 542 | try: 543 | channel=int(m.group(1)) # Radio channel should be an integer 544 | except: 545 | try: 546 | return(None, None, m.group(3).strip()) # Return what the regex thinks is the packet 547 | except: 548 | return(None, None, line.strip()) # Return the whole line 549 | 550 | try: 551 | epoch=int(m.group(2)) # Look for a timestamp 552 | except: 553 | try: 554 | return(channel, None, m.group(3).strip()) # Channel and packet, no time 555 | except: 556 | # Should we check for a packet in group 2? 557 | return(channel, None, line.strip()) # Something probably went wrong; return it anyway because maybe it actually works 558 | 559 | try: 560 | return(channel, epoch, m.group(3).strip()) # It's all there! Wonderful! 561 | except: 562 | return(channel, epoch, line.strip()) # Something is probably wrong, but we'll return what we can and it can fail later 563 | 564 | def get_valid_line(): 565 | """Get a valid line from stdin 566 | returns: boolean: is_valid, string: text""" 567 | is_valid=False 568 | text='a' 569 | try: 570 | text = sys.stdin.readline() 571 | is_valid=True 572 | except UnicodeDecodeError as ude: # Not sure where these come from, but they can't stop the collector 573 | pass 574 | return (is_valid, text) 575 | 576 | 577 | def hex_replace(matchobj): 578 | """Replacement formula for doing regex of non-printing ASCII characters 579 | matchobj: an re.match object, which should be a two-digit hex code 580 | returns: the character corresponding to the two-digit hex code, or null string 581 | """ 582 | if int(matchobj.group(1), 16) != 0: 583 | return(chr(int(matchobj.group(1), 16))) 584 | else: 585 | return("") 586 | 587 | def direwolf_escape(text): 588 | """Escapes direwolf non-printable characters""" 589 | # Use the hex_replace function to determine how to substitute the matched text (i.e. non-printing ASCII) 590 | return(re.sub(r"<0x([0-9A-Fa-f]{2})>", hex_replace, text)) 591 | 592 | 593 | if __name__ == "__main__": # Program is running directly 594 | rxinfo['rx_loc_id'] = check_rx_station(conn, rxinfo) # Test connectivity to database, and get rx_loc_id while we're at it 595 | print("Connection OK") 596 | lastline = 'a' 597 | rxinfo['gps_loc_time'] = time.time() 598 | while lastline != '': # Keep parsing packets from stdin 599 | is_valid=False 600 | while is_valid==False: # Keep trying to parse lines until one is valid 601 | (is_valid, lastline) = get_valid_line() 602 | if lastline.strip()=='q' or lastline.strip()=='2legit': # to quit 603 | exit(0) 604 | lastline = direwolf_escape(lastline) # Catch non-printing ASCII 605 | print(lastline) # DEBUG 606 | 607 | # Parse the direwolf output (channel, timestamp, packet) 608 | (channel, mytime, mypacket) = process_direwolf(lastline) 609 | # Process that output 610 | process_packet(mypacket, conn, rxtime=mytime) 611 | -------------------------------------------------------------------------------- /aprsdb_creation.sql: -------------------------------------------------------------------------------- 1 | CREATE TABLE sessions( 2 | session_id BIGSERIAL PRIMARY KEY, 3 | start_time_utc_s BIGINT, 4 | session_offset BIGINT DEFAULT 0 5 | ); 6 | 7 | CREATE TABLE location( 8 | lid BIGSERIAL PRIMARY KEY, 9 | latitude DOUBLE PRECISION, 10 | longitude DOUBLE PRECISION, 11 | altitude DOUBLE PRECISION, 12 | linestring geometry, 13 | posambiguity VARCHAR(8) 14 | CONSTRAINT loc_geom_point_chk check(st_geometrytype(linestring) = 'ST_Point'::text OR linestring IS NULL) 15 | ); 16 | 17 | CREATE INDEX location_idx ON location USING GIST (linestring); 18 | 19 | CREATE TABLE common( 20 | pid BIGSERIAL PRIMARY KEY, 21 | src VARCHAR(9), 22 | dest VARCHAR(9), 23 | path VARCHAR(64), 24 | via VARCHAR(16), 25 | format VARCHAR(64) NOT NULL, 26 | raw VARCHAR(332) NOT NULL, 27 | rx_loc_id BIGINT REFERENCES location(lid) ON DELETE CASCADE, 28 | rxtime DOUBLE PRECISION, 29 | rxsession BIGINT REFERENCES sessions(session_id) ON DELETE CASCADE, 30 | is_subpacket BOOL DEFAULT False 31 | ); 32 | 33 | CREATE TABLE aprsdb_errs( 34 | eid SERIAL PRIMARY KEY, 35 | rxtime DOUBLE PRECISION, 36 | rxsession BIGINT REFERENCES sessions(session_id) ON DELETE CASCADE, 37 | raw VARCHAR(350) NOT NULL, -- Theoretical packet max is 332 38 | msg VARCHAR(512) 39 | ); 40 | 41 | CREATE TABLE map_entry( 42 | pid BIGSERIAL PRIMARY KEY REFERENCES common(pid) ON DELETE CASCADE, 43 | lid BIGINT REFERENCES location(lid) ON DELETE CASCADE, 44 | symbol CHAR, 45 | symbol_table CHAR, 46 | course FLOAT, 47 | speed FLOAT, 48 | phg VARCHAR(5) 49 | ); 50 | 51 | 52 | CREATE TABLE uncompressed( 53 | pid BIGINT PRIMARY KEY REFERENCES common(pid) ON DELETE CASCADE, 54 | raw_timestamp varchar(24), 55 | messagecapable BOOL, 56 | has_wx BOOL DEFAULT False, 57 | comment VARCHAR(256), 58 | timestamp VARCHAR(24) 59 | ); 60 | 61 | CREATE TABLE compressed( 62 | pid BIGINT PRIMARY KEY REFERENCES common(pid) ON DELETE CASCADE, 63 | radiorange FLOAT, 64 | messagecapable BOOL, 65 | comment VARCHAR(256), 66 | timestamp VARCHAR(24), 67 | raw_timestamp VARCHAR(24), 68 | gpsfixstatus VARCHAR(16), 69 | telemetry VARCHAR(64) 70 | ); 71 | 72 | CREATE TABLE object( 73 | pid BIGINT PRIMARY KEY REFERENCES common(pid) ON DELETE CASCADE, 74 | raw_timestamp VARCHAR(15), 75 | timestamp VARCHAR(15), 76 | object_format VARCHAR(32), 77 | comment VARCHAR(256), 78 | alive BOOL, 79 | object_name VARCHAR(16), 80 | has_wx BOOL DEFAULT False 81 | ); 82 | 83 | CREATE TABLE mic_e( 84 | pid BIGINT PRIMARY KEY REFERENCES common(pid) ON DELETE CASCADE, 85 | mbits VARCHAR(64), 86 | telemetry VARCHAR(64), 87 | comment VARCHAR(256), 88 | daodatumbyte CHAR, 89 | mtype VARCHAR(64) 90 | ); 91 | 92 | CREATE TABLE message( 93 | pid BIGINT PRIMARY KEY REFERENCES common(pid) ON DELETE CASCADE, 94 | msgNo VARCHAR(5), 95 | response VARCHAR(24), 96 | addressee VARCHAR(24), 97 | message_text VARCHAR(128) 98 | ); 99 | 100 | CREATE TABLE status( 101 | pid BIGINT PRIMARY KEY REFERENCES common(pid) ON DELETE CASCADE, 102 | timestamp VARCHAR(24), 103 | raw_timestamp VARCHAR(24), 104 | status VARCHAR(128) 105 | ); 106 | 107 | CREATE TABLE wx( 108 | pid BIGINT PRIMARY KEY REFERENCES common(pid) ON DELETE CASCADE, 109 | wx_raw_timestamp VARCHAR(15), 110 | weather VARCHAR(256), 111 | wind_direction FLOAT, 112 | wind_speed FLOAT, 113 | wind_gust FLOAT, 114 | temperature FLOAT, 115 | rain_1h FLOAT, 116 | rain_24h FLOAT, 117 | rain_since_midnight FLOAT, 118 | humidity FLOAT, 119 | pressure FLOAT, 120 | luminosity FLOAT, 121 | snow FLOAT, 122 | rain_raw FLOAT 123 | ); 124 | 125 | CREATE TABLE thirdparty( 126 | pid BIGINT PRIMARY KEY REFERENCES common(pid), 127 | subpacket VARCHAR(2048), 128 | subpacket_type VARCHAR(32), 129 | subpacket_id BIGINT REFERENCES common(pid) ON DELETE CASCADE 130 | ); 131 | 132 | CREATE TABLE telemetry_message( 133 | pid BIGINT PRIMARY KEY REFERENCES common(pid), 134 | addressee VARCHAR(16), 135 | tPARM VARCHAR(256), 136 | tUNIT VARCHAR(256), 137 | tBITS VARCHAR(256), 138 | tEQNS VARCHAR(256), 139 | title VARCHAR(64) 140 | ); 141 | 142 | 143 | CREATE TABLE digis( 144 | digi_id SERIAL PRIMARY KEY NOT NULL, 145 | call VARCHAR(9) NOT NULL, 146 | aprs_sym CHAR(1), 147 | aprs_table CHAR(1), 148 | loc GEOMETRY, 149 | CONSTRAINT digi_call UNIQUE(call), 150 | CONSTRAINT digi_loc_point_chk CHECK (st_geometrytype(loc) = 'ST_Point'::text OR loc IS NULL) 151 | ); 152 | 153 | CREATE INDEX digi_spatial_idx ON digis USING GIST (loc); 154 | 155 | CREATE TABLE routes( 156 | route_id SERIAL PRIMARY KEY, 157 | src VARCHAR(9) NOT NULL, 158 | dest VARCHAR(9) NOT NULL 159 | ); 160 | 161 | CREATE TABLE paths( 162 | pid BIGINT REFERENCES common(pid) ON DELETE CASCADE, 163 | hop INTEGER, 164 | route_id INTEGER REFERENCES routes(route_id) ON DELETE CASCADE 165 | ); 166 | 167 | CREATE VIEW rf_digi_counts AS 168 | SELECT ROW_NUMBER() OVER (), d1.call, d1.aprs_sym, 169 | d1.aprs_table, COUNT(c1.src), d1.loc 170 | FROM digis AS d1 INNER JOIN common AS c1 ON c1.src=d1.call 171 | WHERE c1.is_subpacket=False 172 | GROUP BY d1.call, d1.aprs_sym, d1.aprs_table, d1.loc 173 | ORDER BY count DESC, d1.call; 174 | 175 | CREATE VIEW wide3 AS 176 | SELECT ROW_NUMBER() OVER (ORDER BY count(c1.src)), 177 | c1.src, COUNT(c1.src), m2.symbol, m2.symbol_table, l3.linestring 178 | FROM common AS c1 179 | INNER JOIN map_entry AS m2 ON c1.pid=m2.pid 180 | INNER JOIN location AS l3 ON m2.lid=l3.lid 181 | WHERE c1.path LIKE '%WIDE3%' 182 | GROUP BY c1.src, l3.linestring, m2.symbol, m2.symbol_table 183 | ORDER BY count(c1.src) DESC, src ASC; 184 | 185 | CREATE VIEW first_hops AS 186 | SELECT ROW_NUMBER() OVER (ORDER BY c1.src), 187 | c1.src, 188 | c1.format, 189 | d1.call AS digi, 190 | ST_SetSRID(ST_MakeLine(d1.loc, l1.linestring), 4326) AS hopline, 191 | ST_DistanceSphere(d1.loc, l1.linestring)/1000 AS dist_km, 192 | COUNT(*) 193 | FROM common AS c1 194 | INNER JOIN map_entry AS e1 ON c1.pid=e1.pid 195 | INNER JOIN location AS l1 ON e1.lid=l1.lid 196 | INNER JOIN paths AS p1 ON p1.pid=c1.pid 197 | INNER JOIN routes AS r1 ON p1.route_id=r1.route_id 198 | INNER JOIN digis AS d1 ON r1.src=d1.call 199 | WHERE p1.hop=1 AND c1.is_subpacket=False 200 | AND c1.src NOT IN (SELECT call FROM digis) 201 | GROUP BY c1.src, c1.format, d1.call, hopline, dist_km 202 | ORDER BY count DESC, c1.src, d1.call, dist_km DESC; 203 | 204 | CREATE VIEW link_stats AS 205 | SELECT ROW_NUMBER() OVER (), 206 | ST_SetSRID(ST_MakeLine(src.loc, dest.loc), 4326), 207 | ST_DistanceSphere(src.loc, dest.loc)/1000 AS dist_km, 208 | src.call AS src, 209 | dest.call AS dest, 210 | COUNT(*) 211 | FROM routes AS r1 INNER JOIN digis AS src ON r1.src = src.call 212 | INNER JOIN digis AS dest ON r1.dest = dest.call 213 | INNER JOIN paths AS p1 ON r1.route_id = p1.route_id 214 | GROUP BY r1.route_id, src.call, dest.call, src.loc, dest.loc; 215 | 216 | CREATE VIEW tx_igate_positions AS 217 | SELECT ROW_NUMBER() OVER (), 218 | d1.call, 219 | c1.src, 220 | c1.format, 221 | l1.linestring, 222 | COUNT(l1.linestring), 223 | ST_DistanceSphere(l1.linestring, d1.loc)/1000 AS dist_km 224 | FROM common AS c1 INNER JOIN thirdparty AS t1 ON c1.pid=t1.subpacket_id 225 | INNER JOIN common AS c2 on t1.pid=c2.pid 226 | INNER JOIN map_entry AS m1 ON c1.pid=m1.pid 227 | INNER JOIN location AS l1 ON m1.lid=l1.lid 228 | INNER JOIN digis AS d1 ON c2.src=d1.call 229 | GROUP BY d1.call, c1.src, c1.format, l1.linestring, ST_DistanceSphere(l1.linestring, d1.loc)/1000; 230 | 231 | CREATE VIEW tx_igate_positions_last_10 AS -- TX-igated positions last 10 min 232 | SELECT ROW_NUMBER() OVER (), 233 | d1.call, 234 | c1.src, 235 | c1.format, 236 | l1.linestring, 237 | COUNT(l1.linestring), 238 | ST_DistanceSphere(l1.linestring, d1.loc)/1000 AS dist_km 239 | FROM common AS c1 INNER JOIN thirdparty AS t1 ON c1.pid=t1.subpacket_id 240 | INNER JOIN common AS c2 on t1.pid=c2.pid 241 | INNER JOIN map_entry AS m1 ON c1.pid=m1.pid 242 | INNER JOIN location AS l1 ON m1.lid=l1.lid 243 | INNER JOIN digis AS d1 ON c2.src=d1.call 244 | WHERE c1.rxtime > (SELECT max(rxtime)-600 FROM common) -- 600 sec = 10 min 245 | GROUP BY d1.call, c1.src, c1.format, l1.linestring, ST_DistanceSphere(l1.linestring, d1.loc)/1000; 246 | 247 | CREATE VIEW tx_igate_positions_last_60 AS -- TX-igated positions last 60 min 248 | SELECT ROW_NUMBER() OVER (), 249 | d1.call, 250 | c1.src, 251 | c1.format, 252 | l1.linestring, 253 | COUNT(l1.linestring), 254 | ST_DistanceSphere(l1.linestring, d1.loc)/1000 AS dist_km 255 | FROM common AS c1 INNER JOIN thirdparty AS t1 ON c1.pid=t1.subpacket_id 256 | INNER JOIN common AS c2 on t1.pid=c2.pid 257 | INNER JOIN map_entry AS m1 ON c1.pid=m1.pid 258 | INNER JOIN location AS l1 ON m1.lid=l1.lid 259 | INNER JOIN digis AS d1 ON c2.src=d1.call 260 | WHERE c1.rxtime > (SELECT max(rxtime)-3600 FROM common) -- 3600 sec = 10 min 261 | GROUP BY d1.call, c1.src, c1.format, l1.linestring, ST_DistanceSphere(l1.linestring, d1.loc)/1000; 262 | 263 | CREATE VIEW tx_igate_counts AS 264 | SELECT ROW_NUMBER() OVER (), 265 | d1.call, 266 | d1.loc, 267 | COUNT(c1.src) 268 | FROM common AS c1 INNER JOIN digis AS d1 on d1.call=c1.src 269 | WHERE c1.format='thirdparty' 270 | GROUP BY d1.call, d1.loc; 271 | 272 | CREATE VIEW rf_objects AS 273 | SELECT row_number() OVER (ORDER BY c1.src, o1.object_name), 274 | o1.object_name AS object_name, 275 | l1.linestring AS linestring, 276 | m1.symbol AS symbol, 277 | m1.symbol_table AS symbol_table, 278 | o1.comment AS comment, 279 | count(o1.object_name), 280 | c1.src AS src, 281 | o1.has_wx AS has_wx 282 | FROM object AS o1 283 | INNER JOIN common AS c1 ON o1.pid=c1.pid 284 | INNER JOIN map_entry AS m1 ON o1.pid=m1.pid 285 | INNER JOIN location AS l1 ON m1.lid=l1.lid 286 | WHERE c1.is_subpacket=False 287 | GROUP BY c1.src, o1.object_name, l1.linestring, m1.symbol, m1.symbol_table, o1.comment, o1.has_wx; 288 | 289 | CREATE VIEW rf_positions AS 290 | SELECT row_number() OVER (), 291 | CASE WHEN c1.format='object' THEN o1.object_name 292 | ELSE c1.src 293 | END AS src, 294 | c1.format, 295 | m1.symbol, 296 | m1.symbol_table, 297 | count(l1.linestring), 298 | l1.linestring 299 | FROM map_entry AS m1 300 | INNER JOIN common AS c1 ON m1.pid=c1.pid 301 | INNER JOIN location AS l1 ON l1.lid=m1.lid 302 | FULL JOIN object AS o1 ON m1.pid=o1.pid 303 | WHERE c1.is_subpacket=False 304 | GROUP BY c1.src, c1.format, m1.symbol, m1.symbol_table, l1.linestring, o1.object_name; 305 | 306 | CREATE VIEW rf_positions_last_10 AS -- RF positions last 10 minutes 307 | SELECT row_number() OVER (), 308 | CASE WHEN c1.format='object' THEN o1.object_name 309 | ELSE c1.src 310 | END AS src, 311 | c1.format, 312 | m1.symbol, 313 | m1.symbol_table, 314 | count(l1.linestring), 315 | l1.linestring 316 | FROM map_entry AS m1 317 | INNER JOIN common AS c1 ON m1.pid=c1.pid 318 | INNER JOIN location AS l1 ON l1.lid=m1.lid 319 | FULL JOIN object AS o1 ON m1.pid=o1.pid 320 | WHERE c1.rxtime > (SELECT max(rxtime)-600 FROM common) -- 600 sec = 10 min 321 | AND c1.is_subpacket=False 322 | GROUP BY c1.src, c1.format, m1.symbol, m1.symbol_table, l1.linestring, o1.object_name; 323 | 324 | CREATE VIEW rf_positions_last_60 AS -- RF positions last 60 minutes 325 | SELECT row_number() OVER (), 326 | CASE WHEN c1.format='object' THEN o1.object_name 327 | ELSE c1.src 328 | END AS src, 329 | c1.format, 330 | m1.symbol, 331 | m1.symbol_table, 332 | count(l1.linestring), 333 | l1.linestring 334 | FROM map_entry AS m1 335 | INNER JOIN common AS c1 ON m1.pid=c1.pid 336 | INNER JOIN location AS l1 ON l1.lid=m1.lid 337 | FULL JOIN object AS o1 ON m1.pid=o1.pid 338 | WHERE c1.rxtime > (SELECT max(rxtime)-3600 FROM common) --3600 sec = 60 min 339 | AND c1.is_subpacket=False 340 | GROUP BY c1.src, c1.format, m1.symbol, m1.symbol_table, l1.linestring, o1.object_name; 341 | 342 | CREATE VIEW digi_stats AS 343 | SELECT d1.*, 344 | coalesce(inbound.count,0) AS heard, 345 | coalesce(outbound.count,0) AS heard_by 346 | FROM digis AS d1 347 | FULL OUTER JOIN 348 | (SELECT r1.src, count(coalesce(r1.src)) 349 | FROM routes AS r1 FULL OUTER JOIN digis AS d1 ON r1.src=d1.call 350 | GROUP BY r1.src) 351 | AS outbound ON d1.call=outbound.src 352 | FULL OUTER JOIN 353 | (SELECT r2.dest, count(coalesce(r2.src)) 354 | FROM routes AS r2 FULL OUTER JOIN digis AS d2 ON r2.dest=d2.call 355 | GROUP BY r2.dest) 356 | AS inbound ON d1.call=inbound.dest; 357 | 358 | CREATE VIEW links_last_10 AS 359 | SELECT ROW_NUMBER() OVER(), 360 | ST_SetSRID(ST_MakeLine(src.loc, dest.loc), 4326), 361 | ST_DistanceSphere(src.loc, dest.loc)/1000 AS dist_km, 362 | src.call AS src, 363 | dest.call AS dest, 364 | COUNT(*) 365 | FROM common AS c1 INNER JOIN paths AS p1 ON c1.pid=p1.pid 366 | INNER JOIN routes AS r1 ON r1.route_id=p1.route_id 367 | INNER JOIN digis AS src ON r1.src=src.call 368 | INNER JOIN digis AS dest ON r1.dest=dest.call 369 | WHERE c1.rxtime > (SELECT max(rxtime)-600 FROM common) 370 | AND c1.is_subpacket=False 371 | GROUP BY r1.route_id, src.call, dest.call, src.loc, dest.loc; 372 | 373 | CREATE VIEW links_last_60 AS 374 | SELECT ROW_NUMBER() OVER(), 375 | ST_SetSRID(ST_MakeLine(src.loc, dest.loc), 4326), 376 | ST_DistanceSphere(src.loc, dest.loc)/1000 AS dist_km, 377 | src.call AS src, 378 | dest.call AS dest, 379 | COUNT(*) 380 | FROM common AS c1 INNER JOIN paths AS p1 ON c1.pid=p1.pid 381 | INNER JOIN routes AS r1 ON r1.route_id=p1.route_id 382 | INNER JOIN digis AS src ON r1.src=src.call 383 | INNER JOIN digis AS dest ON r1.dest=dest.call 384 | WHERE c1.rxtime > (SELECT max(rxtime)-3600 FROM common) 385 | AND c1.is_subpacket=False 386 | GROUP BY r1.route_id, src.call, dest.call, src.loc, dest.loc; 387 | 388 | CREATE VIEW rx_locations AS 389 | SELECT l1.lid, 390 | l1.linestring, 391 | MIN(to_timestamp(c1.rxtime) at time zone 'UTC'), 392 | MAX(to_timestamp(c1.rxtime) at time zone 'UTC'), 393 | COUNT(*) 394 | FROM common AS c1 INNER JOIN location AS l1 ON c1.rx_loc_id=l1.lid 395 | GROUP BY l1.lid; 396 | 397 | 398 | CREATE VIEW heard_digi AS 399 | (SELECT t1.pid, d1.call, 400 | ST_SetSRID(ST_MakeLine(l1.linestring, d1.loc), 4326) AS last_hop, 401 | ST_DistanceSphere(l1.linestring, d1.loc)/1000 AS dist_km 402 | FROM 403 | (SELECT paths.pid, 404 | MAX(paths.hop) AS last_hop 405 | FROM paths 406 | GROUP BY paths.pid) 407 | AS t1 408 | INNER JOIN common AS c1 ON t1.pid=c1.pid 409 | INNER JOIN location AS l1 ON c1.rx_loc_id=l1.lid 410 | INNER JOIN paths AS p1 ON t1.pid = p1.pid 411 | INNER JOIN routes AS r1 ON p1.route_id = r1.route_id 412 | INNER JOIN digis AS d1 ON r1.src = d1.call 413 | WHERE p1.hop=t1.last_hop); 414 | 415 | CREATE VIEW heard_non_digi AS 416 | (SELECT c1.pid, c1.src, 417 | ST_SetSRID(ST_MakeLine(l1.linestring, l2.linestring), 4326) AS last_hop, 418 | ST_DistanceSphere(l1.linestring, l2.linestring)/1000 AS dist_km 419 | FROM common AS c1 420 | INNER JOIN location AS l1 ON c1.rx_loc_id = l1.lid 421 | INNER JOIN map_entry AS m1 ON c1.pid=m1.pid 422 | INNER JOIN location AS l2 ON m1.lid=l2.lid 423 | LEFT JOIN paths AS p1 ON c1.pid=p1.pid 424 | WHERE p1.hop IS NULL AND c1.is_subpacket=false AND c1.format!='object'); 425 | 426 | -------------------------------------------------------------------------------- /aprsgps.py: -------------------------------------------------------------------------------- 1 | # aprsgps.py 2 | # APRS-related GPS functions 3 | 4 | import gpsd 5 | import decimal 6 | 7 | def getLoc2D(): 8 | """Get 2D location (lat/long) from GPS 9 | returns: (latitude, longitude), DD.DDDD format (truncated) """ 10 | gpsd.connect() 11 | myloc = gpsd.get_current() 12 | return((float(round(decimal.Decimal(myloc.lat),4)),float(round(decimal.Decimal(myloc.lon),4)))) 13 | -------------------------------------------------------------------------------- /db_access_privs.sql: -------------------------------------------------------------------------------- 1 | -- Script for handling Postgres user access permissions for APRSDB user moose 2 | REVOKE ALL ON SCHEMA public FROM public; -- Restrict db access (esp. CREATE) 3 | REVOKE ALL ON SCHEMA public FROM moose; 4 | REVOKE ALL ON ALL TABLES IN SCHEMA public FROM moose; 5 | GRANT USAGE ON SCHEMA public TO PUBLIC; 6 | GRANT SELECT ON ALL TABLES IN SCHEMA public TO PUBLIC; 7 | -- Give the new user SELECT privileges on public tables 8 | GRANT USAGE ON SCHEMA public TO moose; 9 | GRANT SELECT ON ALL TABLES IN SCHEMA public TO moose; 10 | GRANT ALL ON DATABASE aprs TO radio; 11 | GRANT ALL ON SCHEMA public TO radio; 12 | GRANT ALL ON ALL TABLES IN SCHEMA public TO radio; 13 | 14 | -------------------------------------------------------------------------------- /db_roles.sql: -------------------------------------------------------------------------------- 1 | -- Create a role (user) with limited (read-only) privileges 2 | CREATE ROLE moose WITH NOSUPERUSER NOCREATEDB NOCREATEROLE NOINHERIT LOGIN NOREPLICATION CONNECTION LIMIT 64 ENCRYPTED PASSWORD 'AE0EE'; 3 | -- Create a role with full read/write privileges but not DB Admin privs 4 | CREATE ROLE radio WITH NOSUPERUSER NOCREATEDB NOCREATEROLE NOINHERIT LOGIN NOREPLICATION CONNECTION LIMIT 64 ENCRYPTED PASSWORD 'aprsPropagation/AE0EE' 5 | 6 | -------------------------------------------------------------------------------- /enable_postgis.sql: -------------------------------------------------------------------------------- 1 | -- Create the aprs database from a superuser account (e.g. postgres) 2 | -- /usr/bin/createdb aprs 3 | 4 | -- Run as admin on aprs database to enable postgis 5 | -- You need only install the features you want 6 | CREATE EXTENSION postgis; 7 | CREATE EXTENSION postgis_topology; 8 | --CREATE EXTENSION postgis_sfcgal; 9 | CREATE EXTENSION fuzzystrmatch; 10 | --CREATE EXTENSION address_standardizer; 11 | --CREATE EXTENSION address_standardizer_data_us; 12 | CREATE EXTENSION postgis_tiger_geocoder; 13 | 14 | -------------------------------------------------------------------------------- /generic_aprsdb.conf: -------------------------------------------------------------------------------- 1 | # config.txt 2 | # Configuration file for aprsdb 3 | 4 | # Receive station info 5 | [aprs] 6 | rxcall = MYCALL 7 | rxsymbol = & 8 | rxtable = \ 9 | latitude = 45 10 | longitude = -93 11 | 12 | # Database connection info for a read-only connection 13 | [psql] 14 | localuser=False 15 | dbname = aprs 16 | dbuser = moose 17 | dbpass = AE0EE 18 | dbhost = 127.0.0.1 19 | dbport = 5432 20 | 21 | # Database connection info for a write-enabled connection 22 | # The user/pass will need to be changed from defaults 23 | # localuser: is the database authentication to be from the local user login? 24 | [psqlw] 25 | localuser=False 26 | dbuser = radio 27 | dbpass = aprsPropagation/AE0EE 28 | dbhost = 127.0.0.1 29 | dbport = 5432 30 | 31 | # Direwolf parsing info 32 | [direwolf] 33 | enable_offline_processing=True 34 | timestamp_format=%Z 35 | -------------------------------------------------------------------------------- /rm_tables.sql: -------------------------------------------------------------------------------- 1 | DROP TABLE sessions CASCADE; 2 | DROP TABLE common CASCADE; 3 | DROP TABLE aprsdb_errs CASCADE; 4 | DROP TABLE location CASCADE; 5 | DROP TABLE map_entry CASCADE; 6 | DROP TABLE uncompressed CASCADE; 7 | DROP TABLE compressed CASCADE; 8 | DROP TABLE object CASCADE; 9 | DROP TABLE mic_e CASCADE; 10 | DROP TABLE message CASCADE; 11 | DROP TABLE status CASCADE; 12 | DROP TABLE wx CASCADE; 13 | DROP TABLE thirdparty CASCADE; 14 | DROP TABLE telemetry_message CASCADE; 15 | DROP TABLE digis CASCADE; 16 | DROP TABLE routes CASCADE; 17 | DROP TABLE paths CASCADE; 18 | DROP VIEW IF EXISTS rf_digi_counts; 19 | DROP VIEW IF EXISTS wide3; 20 | DROP VIEW IF EXISTS first_hops; 21 | DROP VIEW IF EXISTS link_stats; 22 | DROP VIEW IF EXISTS tx_igate_positions; 23 | DROP VIEW IF EXISTS tx_igate_counts; 24 | DROP VIEW IF EXISTS links_last_10; 25 | DROP VIEW IF EXISTS links_last_60; 26 | DROP VIEW IF EXISTS heard_digi; 27 | DROP VIEW IF EXISTS heard_non_digi; 28 | DROP INDEX location_idx; 29 | DROP INDEX digi_spatial_idx; 30 | --------------------------------------------------------------------------------