├── .github ├── ISSUE_TEMPLATE │ ├── bug_report.md │ └── feature_request.md └── PULL_REQUEST_TEMPLATE.md ├── .gitignore ├── .travis.yml ├── .travis_scripts ├── install.sh └── trigger-travis.sh ├── LICENSE ├── README.md ├── app.py ├── config.json ├── configure.py ├── extras ├── connect-error.wav ├── detection-bell.wav ├── error-tada.wav ├── problem.wav └── recognition-error.wav ├── requirements-rpi.txt ├── requirements.txt ├── susi_linux ├── __init__.py ├── __main__.py ├── action_scheduler.py ├── hardware_components │ ├── __init__.py │ ├── led.py │ ├── lights.py │ ├── rpi_wake_button.py │ └── wake_button.py ├── hotword_engine │ ├── Attention.pmdl │ ├── Robot.pmdl │ ├── __init__.py │ ├── computer.pmdl │ ├── hotword_detector.py │ ├── jarvis.pmdl │ ├── snowboy_detector.py │ ├── sphinx_detector.py │ ├── stop.pmdl │ ├── stopMusic.pmdl │ └── susi.pmdl ├── internet_test.py ├── player.py ├── speech │ ├── TTS.py │ └── __init__.py ├── susi_loop.py ├── ui │ ├── __init__.py │ ├── animators.py │ ├── app_window.py │ ├── configuration_window.py │ ├── glade_files │ │ ├── configure.glade │ │ ├── images │ │ │ ├── error.png │ │ │ ├── microphone.png │ │ │ └── susi_icon.png │ │ ├── signin.glade │ │ └── susi_app.glade │ ├── login_window.py │ └── renderer.py └── wav │ ├── daniel_no_connection_to_the_internet.wav │ ├── infobleep.wav │ ├── readme.txt │ ├── ting-ting_no_connection_to_the_internet.wav │ ├── ting-ting_susi_cannot_start.wav │ ├── ting-ting_susi_has_access_to_the_internet.wav │ ├── ting-ting_susi_has_started.wav │ ├── ting-ting_susi_is_alive_and_listening.wav │ └── ting-ting_susi_is_in_startup_mode.wav └── system-integration ├── desktop ├── susi-linux-app.desktop.in └── susi-linux-configure.desktop.in ├── scripts ├── susi-linux ├── susi-linux-app └── susi-linux-configure └── systemd ├── ss-susi-linux.service.in └── ss-susi-linux@.service.in /.github/ISSUE_TEMPLATE/bug_report.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Bug report 3 | about: Create a report to help us improve 4 | 5 | --- 6 | 7 | #### Describe the bug 8 | A clear and concise description of what the bug is. 9 | 10 | #### To Reproduce 11 | Steps to reproduce the behavior: 12 | 1. Go to '...' 13 | 2. Click on '....' 14 | 3. Scroll down to '....' 15 | 4. See error 16 | 17 | #### Expected behavior 18 | A clear and concise description of what you expected to happen. 19 | 20 | #### Screenshots 21 | If applicable, add screenshots to help explain your problem. 22 | 23 | #### Desktop (please complete the following information): 24 | - OS: [e.g. Windows 10, Ubuntu 18.10] 25 | - Browser [e.g. Chrome, Safari] 26 | - Version [e.g. 22] 27 | 28 | #### Smartphone (please complete the following information): 29 | - Device: [e.g. iPhone6] 30 | - OS: [e.g. iOS8.1] 31 | - Browser [e.g. stock browser, Safari] 32 | - Version [e.g. 22] 33 | 34 | #### Additional context 35 | Add any other context about the problem here. 36 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/feature_request.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Feature request 3 | about: Suggest an idea for this project 4 | 5 | --- 6 | 7 | #### Is your feature request related to a problem? Please describe. 8 | A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] 9 | 10 | #### Describe the solution you'd like 11 | A clear and concise description of what you want to happen. 12 | 13 | #### Describe alternatives you've considered 14 | A clear and concise description of any alternative solutions or features you've considered. 15 | 16 | #### Additional context 17 | Add any other context or screenshots about the feature request here. 18 | -------------------------------------------------------------------------------- /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | Fixes # 2 | 3 | #### Checklist 4 | 5 | - [ ] I have read the Contribution & Best practices Guide and my PR follows them. 6 | - [ ] My branch is up-to-date with the Upstream master branch. 7 | - [ ] The unit tests pass locally with my changes 8 | - [ ] I have added tests that prove my fix is effective or that my feature works 9 | - [ ] I have added necessary documentation (if appropriate) 10 | - [ ] All the functions created/modified in this PR contain relevant docstrings. 11 | 12 | #### Test Passing 13 | 14 | - [ ] The SUSI Server must be building on the pi on bootup 15 | - [ ] The hotword detection should have a decent accuracy 16 | - [ ] SUSI Linux shouldn't crash when switching from online to offline and vice versa (failing as of now) 17 | - [ ] SUSI Linux should be able to boot offline when no internet connection available (failing) 18 | 19 | #### Short description of what this resolves: 20 | 21 | #### Changes proposed in this pull request: 22 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | 2 | .idea/ 3 | .vscode 4 | .response 5 | extras/cmu_us_slt.flitevox 6 | requirements.txt 7 | **/__pycache__/ 8 | /susi_python 9 | /hwmixer 10 | /vlcplayer 11 | susi_linux/hotword_engine/snowboy/_snowboydetect.so 12 | susi_linux/hotword_engine/snowboy/snowboydetect.py 13 | /extras/output.wav 14 | /snowboy/ 15 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | sudo: required 2 | dist: trusty 3 | language: python 4 | python: 5 | - "3.5" 6 | jobs: 7 | include: 8 | - stage: first 9 | name: "Trigger rebuild of susi installer" 10 | script: | 11 | echo "TRAVIS_BRANCH=$TRAVIS_BRANCH TRAVIS_PULL_REQUEST=$TRAVIS_PULL_REQUEST" 12 | sh .travis_scripts/trigger-travis.sh --branch development fossasia susi_installer $TRAVIS_ACCESS_TOKEN 13 | 14 | 15 | # 16 | # disabled since it does not work and no tests are available at the moment 17 | # - stage: first # runs in parallel with the previous job 18 | # script: | 19 | # sudo apt-get update --fix-missing 20 | # # We are going to install realpath, which comes as a separate package in Ubuntu Trusty 21 | # sudo apt-get install realpath 22 | # chmod +x .travis_scripts/install.sh 23 | # ./.travis_scripts/install.sh 24 | # pip3 install -U pytest 25 | # pip3 install -U pep8 26 | # - stage: test # runs after the first stop has finished 27 | # script: | 28 | # pep8 susi_linux --ignore E501 --exclude=susi_linux/hotword_engine/snowboy_detector.py 29 | # python3 -m pytest tests/ 30 | -------------------------------------------------------------------------------- /.travis_scripts/install.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | 5 | add_fossasia_repo() { 6 | echo "Set pip repo for root" 7 | if ! sudo test -d /root/.pip; then sudo mkdir /root/.pip; fi 8 | echo -e "[global]\nextra-index-url=https://repo.fury.io/fossasia/" | sudo tee /root/.pip/pip.conf 9 | echo "Set pip repo for current user" 10 | if [ ! -d ~/.config/pip ]; then mkdir -p ~/.config/pip ; fi 11 | echo -e "[global]\nextra-index-url=https://repo.fury.io/fossasia/" > ~/.config/pip/pip.conf 12 | } 13 | 14 | add_debian_repo() { 15 | # Add extra debian repo here, if needed 16 | sudo apt update 17 | } 18 | 19 | install_debian_dependencies() 20 | { 21 | sudo -E apt install -y build-essential python3-pip sox libsox-fmt-all flac pulseaudio libpulse-dev \ 22 | python3-cairo python3-flask mpv flite ca-certificates-java 23 | # We specify ca-certificates-java instead of openjdk-(8/9)-jre-headless, so that it will pull the 24 | # appropriate version of JRE-headless, which can be 8 or 9, depending on ARM6 or ARM7 platform. 25 | } 26 | 27 | function install_seed_voicecard_driver() 28 | { 29 | echo "installing Respeaker Mic Array drivers from source" 30 | git clone https://github.com/respeaker/seeed-voicecard.git 31 | cd seeed-voicecard 32 | sudo ./install.sh 33 | cd .. 34 | #tar czf ~/seeed-voicecard.tar.gz seeed-voicecard 35 | #rm -rf seeed-voicecard 36 | } 37 | 38 | function install_dependencies() 39 | { 40 | install_seed_voicecard_driver 41 | } 42 | 43 | #### Main #### 44 | add_fossasia_repo 45 | add_debian_repo 46 | 47 | echo "Downloading dependency: Susi Python API Wrapper" 48 | if [ ! -d "susi_python" ] 49 | then 50 | git clone https://github.com/fossasia/susi_api_wrapper.git 51 | 52 | echo "setting correct location" 53 | mv susi_api_wrapper/python_wrapper/susi_python susi_python 54 | mv susi_api_wrapper/python_wrapper/requirements.txt requirements.txt 55 | rm -rf susi_api_wrapper 56 | fi 57 | 58 | echo "Installing required Debian Packages" 59 | install_debian_dependencies 60 | 61 | echo "Installing Python Dependencies" 62 | sudo -H pip3 install -U pip wheel 63 | pip3 install -r requirements.txt # This is from susi_api_wrapper 64 | pip3 install -r requirements-hw.txt 65 | pip3 install -r requirements-special.txt 66 | 67 | echo "Downloading Speech Data for flite TTS" 68 | 69 | if [ ! -f "extras/cmu_us_slt.flitevox" ] 70 | then 71 | wget "http://www.festvox.org/flite/packed/flite-2.0/voices/cmu_us_slt.flitevox" -P extras 72 | fi 73 | -------------------------------------------------------------------------------- /.travis_scripts/trigger-travis.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh -f 2 | 3 | # Trigger a new Travis-CI job. 4 | # Ordinarily, a new Travis job is triggered when a commit is pushed to a 5 | # GitHub repository. The trigger-travis.sh script provides a programmatic 6 | # way to trigger a new Travis job. 7 | 8 | # Usage: 9 | # trigger-travis.sh [--pro] [--branch BRANCH] GITHUBID GITHUBPROJECT TRAVIS_ACCESS_TOKEN [MESSAGE] 10 | # For example: 11 | # trigger-travis.sh typetools checker-framework `cat ~/private/.travis-access-token` "Trigger for testing" 12 | # 13 | # where --pro means to use travis-ci.com instead of travis-ci.org, and 14 | # where TRAVIS_ACCESS_TOKEN is, or ~/private/.travis-access-token contains, 15 | # the Travis access token. 16 | # 17 | # Your Travis access token is the text after "Your access token is " in 18 | # the output of this compound command: 19 | # travis login && travis token 20 | # (If the travis program isn't installed, then use either of these two commands: 21 | # gem install travis 22 | # sudo apt-get install ruby-dev && sudo gem install travis 23 | # Don't do "sudo apt-get install travis" which installs a trajectory analyzer.) 24 | # Note that the Travis access token output by `travis token` differs from the 25 | # Travis token available at https://travis-ci.org/profile . 26 | # If you store it in in a file, make sure the file is not readable by others, 27 | # for example by running: chmod og-rwx ~/private/.travis-access-token 28 | 29 | # To use this script to trigger a dependent build in Travis, do two things: 30 | # 31 | # 1. Set an environment variable TRAVIS_ACCESS_TOKEN by navigating to 32 | # https://travis-ci.org/MYGITHUBID/MYGITHUBPROJECT/settings 33 | # The TRAVIS_ACCESS_TOKEN environment variable will be set when Travis runs 34 | # the job, but won't be visible to anyone browsing https://travis-ci.org/. 35 | # 36 | # 2. Add the following to your .travis.yml file, where you replace 37 | # OTHERGITHUB* by a specific downstream project, but you leave 38 | # $TRAVIS_ACCESS_TOKEN as literal text: 39 | # 40 | # jobs: 41 | # include: 42 | # - stage: trigger downstream 43 | # jdk: oraclejdk8 44 | # script: | 45 | # echo "TRAVIS_BRANCH=$TRAVIS_BRANCH TRAVIS_PULL_REQUEST=$TRAVIS_PULL_REQUEST" 46 | # if [[ ($TRAVIS_BRANCH == master) && 47 | # ($TRAVIS_PULL_REQUEST == false) ]] ; then 48 | # curl -LO --retry 3 https://raw.github.com/mernst/plume-lib/master/bin/trigger-travis.sh 49 | # sh trigger-travis.sh OTHERGITHUBID OTHERGITHUBPROJECT $TRAVIS_ACCESS_TOKEN 50 | # fi 51 | 52 | # TODO: Show how to use the --branch command-line argument. 53 | # TODO: Enable the script to clone a particular branch rather than master. 54 | # This would require a way to know the relationships among branches in 55 | # different GitHub projects. It's easier to run all your tests within a 56 | # single Travis job, if they fit within Travis's 50-minute time limit. 57 | 58 | # An alternative to this script would be to install the Travis command-line 59 | # client and then run: 60 | # travis restart -r OTHERGITHUBID/OTHERGITHUBPROJECT 61 | # That is undesirable because it restarts an old job, destroying its history, 62 | # rather than starting a new job which is our goal. 63 | 64 | # Parts of this script were originally taken from 65 | # http://docs.travis-ci.com/user/triggering-builds/ 66 | 67 | 68 | if [ "$#" -lt 3 ] || [ "$#" -ge 7 ]; then 69 | echo "Wrong number of arguments $# to trigger-travis.sh; run like:" 70 | echo " trigger-travis.sh [--pro] [--branch BRANCH] GITHUBID GITHUBPROJECT TRAVIS_ACCESS_TOKEN [MESSAGE]" >&2 71 | exit 1 72 | fi 73 | 74 | if [ "$1" = "--pro" ] ; then 75 | TRAVIS_URL=travis-ci.com 76 | shift 77 | else 78 | TRAVIS_URL=travis-ci.org 79 | fi 80 | 81 | if [ "$1" = "--branch" ] ; then 82 | shift 83 | BRANCH="$1" 84 | shift 85 | else 86 | BRANCH=master 87 | fi 88 | 89 | USER=$1 90 | REPO=$2 91 | TOKEN=$3 92 | if [ $# -eq 4 ] ; then 93 | MESSAGE=",\"message\": \"$4\"" 94 | elif [ -n "$TRAVIS_REPO_SLUG" ] ; then 95 | MESSAGE=",\"message\": \"Triggered by upstream build of $TRAVIS_REPO_SLUG commit "`git rev-parse --short HEAD`"\"" 96 | else 97 | MESSAGE="" 98 | fi 99 | ## For debugging: 100 | # echo "USER=$USER" 101 | # echo "REPO=$REPO" 102 | # echo "TOKEN=$TOKEN" 103 | # echo "MESSAGE=$MESSAGE" 104 | 105 | body="{ 106 | \"request\": { 107 | \"branch\":\"$BRANCH\", 108 | \"config\": { 109 | \"merge_mode\": \"deep_merge\", 110 | \"env\": { 111 | \"global\": { 112 | \"TRIGGER_SOURCE\": \"$TRAVIS_REPO_SLUG\", 113 | \"TRIGGER_BRANCH\": \"$TRAVIS_BRANCH\" 114 | } 115 | } 116 | } 117 | $MESSAGE 118 | }}" 119 | 120 | # It does not work to put / in place of %2F in the URL below. I'm not sure why. 121 | curl -s -X POST \ 122 | -H "Content-Type: application/json" \ 123 | -H "Accept: application/json" \ 124 | -H "Travis-API-Version: 3" \ 125 | -H "Authorization: token ${TOKEN}" \ 126 | -d "$body" \ 127 | https://api.${TRAVIS_URL}/repo/${USER}%2F${REPO}/requests \ 128 | | tee /tmp/travis-request-output.$$.txt 129 | 130 | if grep -q '"@type": "error"' /tmp/travis-request-output.$$.txt; then 131 | exit 1 132 | fi 133 | if grep -q 'access denied' /tmp/travis-request-output.$$.txt; then 134 | exit 1 135 | fi 136 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "{}" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright {yyyy} {name of copyright owner} 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # SUSI.AI on Linux 2 | 3 | [![Codacy Badge](https://api.codacy.com/project/badge/Grade/167b701c744841c5a05269d06b863732)](https://app.codacy.com/app/fossasia/susi_linux?utm_source=github.com&utm_medium=referral&utm_content=fossasia/susi_linux&utm_campaign=badger) 4 | [![Build Status](https://travis-ci.org/fossasia/susi_linux.svg?branch=master)](https://travis-ci.org/fossasia/susi_linux) 5 | [![Join the chat at https://gitter.im/fossasia/susi_hardware](https://badges.gitter.im/fossasia/susi_hardware.svg)](https://gitter.im/fossasia/susi_hardware?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) 6 | [![Twitter Follow](https://img.shields.io/twitter/follow/susiai_.svg?style=social&label=Follow&maxAge=2592000?style=flat-square)](https://twitter.com/susiai_) 7 | 8 | This repository contains components to run SUSI.AI on the desktop or a headless smart speaker together with the [SUSI.AI Server](https://github.com/fossasia/susi_server). Functionalities implemented here include using the microphone to collect voice commands, converting speech to text (STT) using components such as Deep Speech, Flite, Pocket Sphinx, IBM Watson or others, controlling the volume with voice commands and providing a simple GTK interface. In order to use the JSON output of the SUSI.AI Server (written in Java) we use a [SUSI.AI API Python Wrapper](https://github.com/fossasia/susi_python). The ultimate goal of the project to enable users to install SUSI.AI anywhere, apart from desktops and smart speakers on IoT devices, car systems, washing machines and more. 9 | 10 | The functionalities of the project are provided as follows: 11 | 12 | - Hotword detection works for hotword "Susi" 13 | - Voice detection for Speech to Text (STT) using with Google Speech API, IBM Watson Speech to Text API 14 | - Voice output for Text to Speech (TTS) working with Google Voice, IBM Watson TTS, Flite TTS 15 | - SUSI.AI response working through [SUSI.AI API Python Wrapper](https://github.com/fossasia/susi_python) 16 | 17 | ## Project Overview 18 | 19 | The SUSI.AI ecosystem consists of the following parts: 20 | ``` 21 | * Web Client and Content Management System for the SUSI.AI Skills - Home of the SUSI.AI community 22 | |_ susi.ai (React Application, User Account Management for the CMS, a client for the susi_server at https://api.susi.ai the content management system for susi skills) 23 | 24 | * server back-end 25 | |_ susi_server (the brain of the infrastructure, a server which computes answers from queries) 26 | |_ susi_skill_data (the knowledge of the brain, a large collection of skills provided by the SUSI.AI community) 27 | 28 | * android front-end 29 | |_ susi_android (Android application which is a client for the susi_server at https://api.susi.ai) 30 | 31 | * iOS front-end 32 | |_ susi_iOS (iOS application which is a client for the susi_server at https://api.susi.ai) 33 | 34 | * Smart Speaker - Software to turn a Raspberry Pi into a Personal Assistant 35 | | Several sub-projects come together in this device 36 | |_ susi_installer (Framework which can install all parts on a RPi and Desktops, and also is able to create SUSIbian disk images) 37 | |_ susi_python (Python API for the susi_server at https://api.susi.ai or local instance) 38 | |_ susi_server (The same server as on api.susi.ai, hosted locally for maximum privacy. No cloud needed) 39 | |_ susi_skill_data (The skills as provided by susi_server on api.susi.ai; pulled from the git repository automatically) 40 | |_ susi_linux (a state machine in python which uses susi_python, Speech-to-text and Text-to-speech functions) 41 | |_ susi.ai (React Application, the local web front-end with User Account Management, a client for the local deployment of the susi_server, the content management system for susi skills) 42 | ``` 43 | 44 | ## Installation 45 | 46 | `susi_linux` is normally installed via the [SUSI Installer](https://github.com/fossasia/susi_installer). 47 | In this case there are binaries for configuration and starting and 48 | others available in `$HOME/SUSI.AI/bin` (under default installation settings). 49 | 50 | In case of manual installations, the wrappers in [`wrapper` directory](wrapper/) need to 51 | be configured to point to the respective installation directories and location of 52 | the `config.json` file. 53 | 54 | ## Setting up and configuring Susi on Linux / RaspberryPi 55 | 56 | Configuration is done via the file [config.json](config.json) which normally 57 | resides in `$HOME/.config/SUSI.AI/config.json`. 58 | 59 | The script `$HOME/SUSI.AI/bin/susi-config` is best used to query, set, and 60 | change configuration of `susi_linux`. There is also a GUI interface to the 61 | configuration in `$HOME/SUSI.AI/bin/susi-linux-configure`. 62 | 63 | The possible keys and values are given by running `$HOME/SUSI.AI/bin/susi-config keys` 64 | 65 | Some important keys and possible values: 66 | 67 | ``` 68 | - `stt` is the speech to text service, one of the following choices: 69 | - `google` - use Google STT service 70 | - `watson` - IBM/Watson STT 71 | - `bing` - MS Bing STT 72 | - `pocketsphinx` - PocketSphinx STT system, working offline 73 | - `deepspeech-local` - DeepSpeech STT system, offline, WORK IN PROGRESS 74 | - `tts` is the text to speech service, one of the following choices: 75 | - `google` - use Google TTS 76 | - `watson` - IBM/Watson TTS (login credential necessary) 77 | - `flite` - flite TTS service working offline 78 | - `hotword.engine` is the choice if you want to use snowboy detector as the hotword detection or not 79 | - `Snowboy` to use snowboy 80 | - `PocketSphinx` to use Pocket Sphinx 81 | - `wakebutton` is the choice if you want to use an external wake button or not 82 | - `enabled` to use an external wake button 83 | - `disabled` to disable the external wake button 84 | - `not available` for systems without dedicated wake button 85 | 86 | Other interfaces for configuration are available for Android and iOS. 87 | 88 | Manual configuration is possible, the allowed keys in [`config.json`](config.json) are currently 89 | - `device`: the name of the current device 90 | - `wakebutton`: whether a wake button is available or not 91 | - `stt`: see above for possible settings 92 | - `tts`: see above for possible settings 93 | - `language': language for STT and TTS processing 94 | - `path.base`: directory where support files are installed 95 | - `path.sound.detection`: sound file that is played when detection starts, relative to `data_base_dir` 96 | - `path.sound.problem`: sound file that is played on general errors, relative to `data_base_dir` 97 | - `path.sound.error.recognition`: sound file that is played on detection errors, relative to `data_base_dir` 98 | - `path.sound.error.timeout`: sound file that is played when timing out waiting for spoken commands 99 | - `path.flite_speech`: flitevox speech file, relative to `data_base_dir` 100 | - `hotword.engine`: see above for possible settings 101 | - `hotword.model`: (if hotword.engine = Snowboy) selects the model file for the hotword 102 | - `susi.mode`: access mode to `accounts.susi.ai`, either `anonymous` or `authenticated` 103 | - `susi.user`: (if susi.mode = authenticated) the user name (email) to be used 104 | - `susi.pass`: (if susi.mode = authenticated) the password to be used 105 | - `roomname`: free form description of the room 106 | - `watson.stt.user`, `watson.stt.pass`, `watson.tts.user`, `watson.tts.pass`: credentials for IBM/Watson server for TTS and STT 107 | - `watson.tts.voice`: voice name selected for IBM/Watson TTS 108 | - `bing.api`: Bing STT API key 109 | 110 | For details concerning installation, setup, and operation on RaspberryPi, see 111 | the documentation at [SUSI Installer](https://github.com/fossasia/susi_installer). 112 | 113 | 114 | 115 | ## Information for developers 116 | 117 | This section is intended for developer. 118 | 119 | ### **Important:** Tests before making a new release 120 | 121 | 1. The hotword detection should have a decent accuracy 122 | 2. SUSI Linux shouldn't crash when switching from online to offline and vice versa (failing as of now) 123 | 3. SUSI Linux should be able to boot offline when no internet connection available (failing as of now) 124 | 125 | ### Roadmap 126 | 127 | - Offline Voice Detection (if possible with satisfactory results) 128 | 129 | ### General working of SUSI 130 | 131 | - SUSI.AI follows a finite state system for the code architecture. 132 | - Google TTS and STT services are used as default services but if the internet fails, a switch to offline services PocketSphinx (STT) and Flite (TTS) is made automatically 133 | 134 | 135 | ### Run SUSI Linux for development purposes 136 | 137 | If installed via the SUSI Installer, systemd unit files are installed: 138 | - `ss-susi-linux.service` for the user bus, use as user with `systemctl --user start/enable ss-susi-linux` 139 | - `ss-susi-linux@.service` for the system bus, use as `root` user to start a job for a specific user, 140 | independent from whether the user is logged in or not: `sudo systemctl start/enable ss-susi-linux@USER` 141 | 142 | By default, it is ran in _production_ mode, where log messages are limited to _error_ and _warning_ only. 143 | In development, you may want to see more logs, to help debugging. You can switch it to "verbose" mode by 2 ways: 144 | 145 | 1. Run it manually 146 | 147 | - Stop systemd service by `sudo systemctl stop ss-susi-linux` 148 | - Use Terminal, _cd_ to `susi_linux` directory and run 149 | 150 | ``` 151 | python3 -m susi_linux -v 152 | ``` 153 | or repeat `v` to increase verbosity: 154 | 155 | ``` 156 | python3 -m susi_linux -vv 157 | ``` 158 | 159 | 2. Change command run by `systemd` 160 | 161 | - Edit the _/lib/systemd/system/ss-susi-linux.service_ and change the command in `ExecStart` parameter: 162 | 163 | ```ini 164 | ExecStart=/usr/bin/python3 -m susi_linux -v --short-log 165 | ``` 166 | - Reload systemd daemon: `sudo systemctl daemon-reload` 167 | - Restart the servive: `sudo systemctl restart ss-susi-linux` 168 | - Now you can read the log via `journalctl`: 169 | 170 | + `journalctl -u ss-susi-linux` 171 | + or `journalctl -fu ss-susi-linux` to get updated when the log is continuously produced. 172 | 173 | The `-v` option is actually the same as the 1st method. The `--short-log` option is to exclude some info which is already provided by `journalctl`. For more info about `logging` feature, see this GitHub [issue](https://github.com/fossasia/susi_linux/issues/423). 174 | 175 | -------------------------------------------------------------------------------- /app.py: -------------------------------------------------------------------------------- 1 | from susi_linux.ui import SusiAppWindow 2 | 3 | window = SusiAppWindow() 4 | window.show_window() 5 | -------------------------------------------------------------------------------- /config.json: -------------------------------------------------------------------------------- 1 | { 2 | "Device": "RaspberryPi", 3 | "WakeButton": "not available", 4 | "default_stt": "google", 5 | "default_tts": "google", 6 | "data_base_dir": ".", 7 | "detection_bell_sound": "extras/detection-bell.wav", 8 | "problem_sound": "extras/problem.wav", 9 | "recognition_error_sound": "extras/recognition-error.wav", 10 | "timeout_error_sound": "extras/error-tada.wav", 11 | "flite_speech_file_path": "extras/cmu_us_slt.flitevox", 12 | "hotword_engine": "Snowboy", 13 | "usage_mode": "anonymous", 14 | "room_name": "livingRoom", 15 | "watson_tts_config": { 16 | "username": "", "password": "" 17 | } 18 | } 19 | -------------------------------------------------------------------------------- /configure.py: -------------------------------------------------------------------------------- 1 | from susi_linux.ui import ConfigurationWindow 2 | 3 | window = ConfigurationWindow() 4 | window.show_window() 5 | -------------------------------------------------------------------------------- /extras/connect-error.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/extras/connect-error.wav -------------------------------------------------------------------------------- /extras/detection-bell.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/extras/detection-bell.wav -------------------------------------------------------------------------------- /extras/error-tada.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/extras/error-tada.wav -------------------------------------------------------------------------------- /extras/problem.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/extras/problem.wav -------------------------------------------------------------------------------- /extras/recognition-error.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/extras/recognition-error.wav -------------------------------------------------------------------------------- /requirements-rpi.txt: -------------------------------------------------------------------------------- 1 | RPi.GPIO 2 | spidev 3 | 4 | # Repo for wheel packages, pre-compiled for ARM 5 | --extra-index-url https://repo.fury.io/fossasia/ 6 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | async_promises 2 | colorlog 3 | google_speech 4 | json_config 5 | pafy 6 | pocketsphinx==0.1.15 7 | pyalsaaudio==0.8.4 8 | pyaudio 9 | python-Levenshtein 10 | python-vlc 11 | requests_futures 12 | rx>=3.0.0a0 13 | service_identity 14 | snowboy==1.3.0 15 | speech-recognition-fork>=3.8.1.2020.12.6 16 | watson-developer-cloud 17 | websocket-server 18 | youtube-dl>=2019.6.21 19 | 20 | # Repo for wheel packages, pre-compiled for ARM 21 | --extra-index-url https://repo.fury.io/fossasia/ 22 | -------------------------------------------------------------------------------- /susi_linux/__init__.py: -------------------------------------------------------------------------------- 1 | """ Main Module of the SUSI Linux App 2 | """ 3 | from .susi_loop import SusiLoop 4 | -------------------------------------------------------------------------------- /susi_linux/__main__.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | import logging 4 | import argparse 5 | 6 | import colorlog 7 | 8 | from . import SusiLoop 9 | from .player import player 10 | 11 | 12 | parser = argparse.ArgumentParser(prog='python3 -m susi_linux', 13 | description='SUSI Linux main program') 14 | 15 | 16 | def get_colorlog_handler(short=False): 17 | # Short log format is for use under systemd. 18 | # Here we exclude some info, because they will be added by journalctl. 19 | if short: 20 | log_format = '%(log_color)s%(levelname)s:%(reset)s %(message)s' 21 | else: 22 | log_format = '%(log_color)s%(asctime)s %(levelname)s:%(name)s:%(reset)s %(message)s' 23 | handler = colorlog.StreamHandler() 24 | handler.setFormatter( 25 | colorlog.TTYColoredFormatter( 26 | log_format, 27 | stream=sys.stderr, 28 | datefmt='%Y-%m-%d %H:%M:%S')) 29 | return handler 30 | 31 | 32 | def startup_sound(): 33 | curr_folder = os.path.dirname(os.path.abspath(__file__)) 34 | audio_file = os.path.join(curr_folder, 'wav/ting-ting_susi_has_started.wav') 35 | player.say(audio_file) 36 | 37 | 38 | if __name__ == '__main__': 39 | 40 | parser.add_argument('-v', '--verbose', action='count', default=0, 41 | help='Show log. Repeat to get more detailed one.') 42 | 43 | ''' 44 | Sometimes, when we enable -v in systemd service command, and read the log via 45 | journalctl, we will see duplication of timestamp and process. These info are 46 | provided by both journalctl and our app. Enable --short-log to stop our app 47 | from including those info in log. 48 | ''' 49 | 50 | parser.add_argument('--short-log', action='store_true', 51 | help='Produce log w/o timestamp and process name.') 52 | 53 | args = parser.parse_args() 54 | 55 | # Configure logger 56 | if args.verbose: 57 | levels = (logging.WARNING, logging.INFO, logging.DEBUG) 58 | handler = get_colorlog_handler(args.short_log) 59 | lindex = min(args.verbose, len(levels) - 1) 60 | level = levels[lindex] 61 | # logging.root.propagate = True 62 | logging.root.setLevel(level) 63 | logging.root.handlers = [] 64 | logging.root.addHandler(handler) 65 | 66 | susi_loop = SusiLoop() 67 | startup_sound() 68 | susi_loop.start() 69 | 70 | -------------------------------------------------------------------------------- /susi_linux/action_scheduler.py: -------------------------------------------------------------------------------- 1 | 2 | import time 3 | import sched 4 | from threading import Thread 5 | from rx.subject import Subject 6 | 7 | 8 | class ActionScheduler(Thread): 9 | 10 | def __init__(self): 11 | super().__init__() 12 | self.subject = Subject() 13 | self.events = {} 14 | self.counter = -1 15 | self.scheduler = sched.scheduler(time.time, time.sleep) 16 | 17 | def on_detected(self, reply): 18 | self.subject.on_next(reply) 19 | 20 | def add_event(self, delay, time, reply): 21 | if delay > 0: 22 | self.events[self.counter + 1] = self.scheduler.enter(delay, 0, self.on_detected, argument=(reply,)) 23 | else: 24 | self.events[self.counter + 1] = self.scheduler.enterabs(time, 0, self.on_detected, argument=(reply,)) 25 | self.counter += 1 26 | 27 | def run(self): 28 | while True: 29 | self.scheduler.run(blocking=True) 30 | time.sleep(1) 31 | return 32 | -------------------------------------------------------------------------------- /susi_linux/hardware_components/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/susi_linux/hardware_components/__init__.py -------------------------------------------------------------------------------- /susi_linux/hardware_components/led.py: -------------------------------------------------------------------------------- 1 | try: 2 | import spidev 3 | except ImportError: 4 | print("No spidev, probably no raspi ...") 5 | import subprocess 6 | import sys 7 | import os 8 | from math import ceil 9 | 10 | RGB_MAP = {'rgb': [3, 2, 1], 'rbg': [3, 1, 2], 'grb': [ 11 | 2, 3, 1], 'gbr': [2, 1, 3], 'brg': [1, 3, 2], 'bgr': [1, 2, 3]} 12 | 13 | 14 | class LED_COLOR: 15 | 16 | # Constants 17 | MAX_BRIGHTNESS = 0b11111 # Safeguard: Set to a value appropriate for your setup 18 | LED_START = 0b11100000 # Three "1" bits, followed by 5 brightness bits 19 | 20 | def __init__(self, num_led, global_brightness=MAX_BRIGHTNESS, 21 | order='rgb', bus=0, device=1, max_speed_hz=8000000): 22 | if (os.access("/proc/asound/cards", os.R_OK)): 23 | output = subprocess.check_output( 24 | ["cat", "/proc/asound/cards"]).decode(sys.stdout.encoding) 25 | self.seeed_attached = output.find("seeed") != -1 26 | else: 27 | self.seeed_attached = False 28 | if (not self.seeed_attached): 29 | return 30 | self.num_led = num_led # The number of LEDs in the Strip 31 | order = order.lower() 32 | self.rgb = RGB_MAP.get(order, RGB_MAP['rgb']) 33 | # Limit the brightness to the maximum if it's set higher 34 | if global_brightness > self.MAX_BRIGHTNESS: 35 | self.global_brightness = self.MAX_BRIGHTNESS 36 | else: 37 | self.global_brightness = global_brightness 38 | 39 | self.leds = [self.LED_START, 0, 0, 0] * self.num_led # Pixel buffer 40 | self.spi = spidev.SpiDev() # Init the SPI device 41 | self.spi.open(bus, device) # Open SPI port 0, slave device (CS) 1 42 | # Up the speed a bit, so that the LEDs are painted faster 43 | if max_speed_hz: 44 | self.spi.max_speed_hz = max_speed_hz 45 | 46 | def clock_start_frame(self): 47 | if (not self.seeed_attached): 48 | return 49 | """Sends a start frame to the LED strip. 50 | """ 51 | self.spi.xfer2([0] * 4) # Start frame, 32 zero bits 52 | 53 | def clock_end_frame(self): 54 | if (not self.seeed_attached): 55 | return 56 | self.spi.xfer2([0xFF] * 4) 57 | 58 | # Round up num_led/2 bits (or num_led/16 bytes) 59 | # for _ in range((self.num_led + 15) // 16): 60 | # self.spi.xfer2([0x00]) 61 | 62 | def clear_strip(self): 63 | if (not self.seeed_attached): 64 | return 65 | """ Turns off the strip and shows the result right away.""" 66 | 67 | for led in range(self.num_led): 68 | self.set_pixel(led, 0, 0, 0) 69 | self.show() 70 | 71 | def set_pixel(self, led_num, red, green, blue, bright_percent=100): 72 | if (not self.seeed_attached): 73 | return 74 | """Sets the color of one pixel in the LED stripe. 75 | 76 | The changed pixel is not shown yet on the Stripe, it is only 77 | written to the pixel buffer. Colors are passed individually. 78 | If brightness is not set the global brightness setting is used. 79 | """ 80 | if led_num < 0: 81 | return # Pixel is invisible, so ignore 82 | if led_num >= self.num_led: 83 | return # again, invisible 84 | 85 | # Calculate pixel brightness as a percentage of the 86 | # defined global_brightness. Round up to nearest integer 87 | # as we expect some brightness unless set to 0 88 | brightness = int(ceil(bright_percent * self.global_brightness / 100.0)) 89 | 90 | # LED startframe is three "1" bits, followed by 5 brightness bits 91 | ledstart = (brightness & 0b00011111) | self.LED_START 92 | 93 | start_index = 4 * led_num 94 | self.leds[start_index] = ledstart 95 | self.leds[start_index + self.rgb[0]] = red 96 | self.leds[start_index + self.rgb[1]] = green 97 | self.leds[start_index + self.rgb[2]] = blue 98 | 99 | def set_pixel_rgb(self, led_num, rgb_color, bright_percent=100): 100 | if (not self.seeed_attached): 101 | return 102 | """ 103 | Sets the color of one pixel in the LED stripe. 104 | 105 | The changed pixel is not shown yet on the Stripe, it is only 106 | written to the pixel buffer. 107 | Colors are passed combined (3 bytes concatenated) 108 | If brightness is not set the global brightness setting is used. 109 | """ 110 | self.set_pixel(led_num, (rgb_color & 0xFF0000) >> 16, 111 | (rgb_color & 0x00FF00) >> 8, rgb_color & 0x0000FF, bright_percent) 112 | 113 | def rotate(self, positions=1): 114 | if (not self.seeed_attached): 115 | return 116 | """ 117 | Rotate the LEDs by the specified number of positions. 118 | 119 | Treating the internal LED array as a circular buffer, rotate it by 120 | the specified number of positions. The number could be negative, 121 | which means rotating in the opposite direction. 122 | """ 123 | cutoff = 4 * (positions % self.num_led) 124 | self.leds = self.leds[cutoff:] + self.leds[:cutoff] 125 | 126 | def show(self): 127 | if (not self.seeed_attached): 128 | return 129 | """ 130 | Sends the content of the pixel buffer to the strip. 131 | 132 | Todo: More than 1024 LEDs requires more than one xfer operation. 133 | """ 134 | self.clock_start_frame() 135 | # xfer2 kills the list, unfortunately. So it must be copied first 136 | # SPI takes up to 4096 Integers. So we are fine for up to 1024 LEDs. 137 | data = list(self.leds) 138 | while data: 139 | self.spi.xfer2(data[:32]) 140 | data = data[32:] 141 | self.clock_end_frame() 142 | 143 | def cleanup(self): 144 | if (not self.seeed_attached): 145 | return 146 | """Release the SPI device; Call this method at the end""" 147 | self.spi.close() # Close SPI port 148 | 149 | @staticmethod 150 | def combine_color(red, green, blue): 151 | """Make one 3 * 8 byte color value.""" 152 | 153 | return (red << 16) + (green << 8) + blue 154 | 155 | def wheel(self, wheel_pos): 156 | """Get a color from a color wheel; Green -> Red -> Blue -> Green""" 157 | 158 | if wheel_pos > 255: 159 | wheel_pos = 255 # Safeguard 160 | if wheel_pos < 85: # Green -> Red 161 | return self.combine_color(wheel_pos * 3, 255 - wheel_pos * 3, 0) 162 | if wheel_pos < 170: # Red -> Blue 163 | wheel_pos -= 85 164 | return self.combine_color(255 - wheel_pos * 3, 0, wheel_pos * 3) 165 | # Blue -> Green 166 | wheel_pos -= 170 167 | return self.combine_color(0, wheel_pos * 3, 255 - wheel_pos * 3) 168 | -------------------------------------------------------------------------------- /susi_linux/hardware_components/lights.py: -------------------------------------------------------------------------------- 1 | from .led import * 2 | import time 3 | import threading 4 | try: 5 | import queue as Queue 6 | except ImportError: 7 | import Queue as Queue 8 | 9 | 10 | class Lights: 11 | LIGHTS_N = 3 12 | 13 | def __init__(self): 14 | self.basis = [0] * 3 * self.LIGHTS_N 15 | self.basis[0] = 2 16 | self.basis[3] = 1 17 | self.basis[4] = 1 18 | self.basis[7] = 2 19 | 20 | self.colors = [0] * 3 * self.LIGHTS_N 21 | self.dev = LED_COLOR(num_led=self.LIGHTS_N) 22 | 23 | self.next = threading.Event() 24 | self.queue = Queue.Queue() 25 | self.thread = threading.Thread(target=self._run) 26 | self.thread.daemon = True 27 | self.thread.start() 28 | 29 | def wakeup(self, direction=0): 30 | def f(): 31 | self._wakeup(direction) 32 | 33 | self.next.set() 34 | self.queue.put(f) 35 | 36 | def listen(self): 37 | self.next.set() 38 | self.queue.put(self._listen) 39 | 40 | def think(self): 41 | self.next.set() 42 | self.queue.put(self._think) 43 | 44 | def speak(self): 45 | self.next.set() 46 | self.queue.put(self._speak) 47 | 48 | def off(self): 49 | self.next.set() 50 | self.queue.put(self._off) 51 | 52 | def _run(self): 53 | while True: 54 | func = self.queue.get() 55 | func() 56 | 57 | def _wakeup(self, direction=0): 58 | for i in range(1, 25): 59 | colors = [i * v for v in self.basis] 60 | self.write(colors) 61 | time.sleep(0.01) 62 | 63 | self.colors = colors 64 | 65 | def _listen(self): 66 | for i in range(1, 25): 67 | colors = [i * v for v in self.basis] 68 | self.write(colors) 69 | time.sleep(0.01) 70 | 71 | self.colors = colors 72 | 73 | def _think(self): 74 | colors = self.colors 75 | 76 | self.next.clear() 77 | while not self.next.is_set(): 78 | colors = colors[3:] + colors[:3] 79 | self.write(colors) 80 | time.sleep(0.2) 81 | 82 | t = 0.1 83 | for i in range(0, 5): 84 | colors = colors[3:] + colors[:3] 85 | self.write([(v * (4 - i) / 4) for v in colors]) 86 | time.sleep(t) 87 | t /= 2 88 | 89 | # time.sleep(0.5) 90 | 91 | self.colors = colors 92 | 93 | def _speak(self): 94 | colors = self.colors 95 | gradient = -1 96 | position = 24 97 | 98 | self.next.clear() 99 | while not self.next.is_set(): 100 | position += gradient 101 | self.write([(v * position / 24) for v in colors]) 102 | 103 | if position == 24 or position == 4: 104 | gradient = -gradient 105 | time.sleep(0.2) 106 | else: 107 | time.sleep(0.01) 108 | 109 | while position > 0: 110 | position -= 1 111 | self.write([(v * position / 24) for v in colors]) 112 | time.sleep(0.01) 113 | 114 | # self._off() 115 | 116 | def _off(self): 117 | self.write([0] * 3 * self.LIGHTS_N) 118 | 119 | def write(self, colors): 120 | for i in range(self.LIGHTS_N): 121 | self.dev.set_pixel(i, int(colors[3 * i]), int(colors[3 * i + 1]), int(colors[3 * i + 2])) 122 | 123 | self.dev.show() 124 | 125 | 126 | lights = Lights() 127 | -------------------------------------------------------------------------------- /susi_linux/hardware_components/rpi_wake_button.py: -------------------------------------------------------------------------------- 1 | import RPi.GPIO as GPIO 2 | from .wake_button import WakeButton 3 | 4 | 5 | class RaspberryPiWakeButton(WakeButton): 6 | 7 | def __init__(self): 8 | super().__init__() 9 | GPIO.setmode(GPIO.BCM) 10 | GPIO.setup(17, GPIO.IN, pull_up_down=GPIO.PUD_UP) 11 | GPIO.add_event_detect( 12 | 17, GPIO.FALLING, 13 | callback=self.button_detected, 14 | bouncetime=300) 15 | 16 | def button_detected(channel, foo): 17 | super().on_detected() 18 | 19 | def run(self): 20 | pass 21 | -------------------------------------------------------------------------------- /susi_linux/hardware_components/wake_button.py: -------------------------------------------------------------------------------- 1 | from abc import ABC, abstractclassmethod 2 | from threading import Thread 3 | from rx.subject import Subject 4 | 5 | 6 | class WakeButton(ABC, Thread): 7 | 8 | def __init__(self): 9 | super().__init__() 10 | self.subject = Subject() 11 | 12 | @abstractclassmethod 13 | def run(self): 14 | pass 15 | 16 | def on_detected(self): 17 | self.subject.on_next("Hotword") 18 | -------------------------------------------------------------------------------- /susi_linux/hotword_engine/Attention.pmdl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/susi_linux/hotword_engine/Attention.pmdl -------------------------------------------------------------------------------- /susi_linux/hotword_engine/Robot.pmdl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/susi_linux/hotword_engine/Robot.pmdl -------------------------------------------------------------------------------- /susi_linux/hotword_engine/__init__.py: -------------------------------------------------------------------------------- 1 | """ 2 | This module defines all the hotword detection engines present in the app. 3 | Presently, it support 4 | * PocketSphinx KeyPhrase Search for Hotword Detection 5 | * Snowboy Hotword Detection 6 | 7 | While Snowboy gives marginally better results, if it is unavailable on your device, 8 | you may use PocketSphinx 9 | """ 10 | 11 | import logging 12 | 13 | SNOWBOY_AVAILABLE = False 14 | POCKETSPHINX_AVAILABLE = False 15 | logger = logging.getLogger(__name__) 16 | 17 | 18 | try: 19 | from snowboy.snowboydetect import SnowboyDetect 20 | SNOWBOY_AVAILABLE = True 21 | except ImportError: 22 | pass 23 | 24 | try: 25 | from .sphinx_detector import PocketSphinxDetector 26 | POCKETSPHINX_AVAILABLE = True 27 | except ImportError: 28 | pass 29 | 30 | if SNOWBOY_AVAILABLE is True: 31 | logger.info("Snowboy successfully imported.") 32 | else: 33 | logger.info("Snowboy not currently installed. You may use PocketSphinx instead or you need to install it from https://github.com/Kitt-AI/snowboy.") 34 | 35 | if POCKETSPHINX_AVAILABLE is True: 36 | logger.info("PocketSphinx successfully imported.") 37 | else: 38 | logger.info("PocketSphinx is not currently installed. You may use Snowboy instead.") 39 | 40 | if SNOWBOY_AVAILABLE is True and POCKETSPHINX_AVAILABLE is True: 41 | logger.info("Both Snowboy and PocketSphinx successfully imported. We will recommend using Snowboy.") 42 | -------------------------------------------------------------------------------- /susi_linux/hotword_engine/computer.pmdl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/susi_linux/hotword_engine/computer.pmdl -------------------------------------------------------------------------------- /susi_linux/hotword_engine/hotword_detector.py: -------------------------------------------------------------------------------- 1 | """ 2 | This module defines an abstract class for Hotword Detection. 3 | Any Hotword Detection Engine implemented in the app must inherit 4 | from this class. HotwordDetector subclasses from threading. 5 | Thread since Hotword Detection will run in a separate thread for non-blocking 6 | operation. 7 | """ 8 | 9 | from abc import ABC, abstractclassmethod 10 | from threading import Thread 11 | from rx.subject import Subject 12 | 13 | 14 | class HotwordDetector(ABC, Thread): 15 | """ 16 | This is an abstract class for a Hotword Detector. Any hotword detector 17 | implemented in the app must inherit this. It subclasses from threading. 18 | Thread allowing Hotword Detection to run on a separate thread. 19 | :attributes 20 | callback_queue : A queue to send callbacks to main thread. 21 | detection_callback: A callback function to be called on the calling 22 | thread when hotword is detected. 23 | is_active: A boolean to indicate if hotword detection is currently 24 | active. If inactive, the engine ignores all the hotword detected 25 | in that time. 26 | """ 27 | 28 | def __init__(self) -> None: 29 | Thread.__init__(self) 30 | self.subject = Subject() 31 | 32 | @abstractclassmethod 33 | def run(self): 34 | """ 35 | This method is executed on the start of the thread. You may initialize 36 | parameters for Hotword Detection here and start the recognition in a 37 | busy/wait loop since operation is being run on background thread. 38 | On detecting a hotword, it should call on_detected. 39 | """ 40 | pass 41 | 42 | def on_detected(self): 43 | """ 44 | This callback is fired when a Hotword Detector detects a hotword. 45 | :return: None 46 | """ 47 | self.subject.on_next("Hotword") 48 | 49 | def start(self): 50 | pass 51 | 52 | def stop(self): 53 | pass 54 | 55 | -------------------------------------------------------------------------------- /susi_linux/hotword_engine/jarvis.pmdl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/susi_linux/hotword_engine/jarvis.pmdl -------------------------------------------------------------------------------- /susi_linux/hotword_engine/snowboy_detector.py: -------------------------------------------------------------------------------- 1 | """ 2 | Implementation of SnowboyDetector using Snowboy Hotword Detection Engine. 3 | It provides excellent recognition of Hotword but all devices are not 4 | supported presently. Use PocketSphinx if you face errors with this Detector. 5 | """ 6 | import os 7 | import logging 8 | from .hotword_detector import HotwordDetector 9 | from snowboy import snowboydecoder 10 | 11 | logger = logging.getLogger(__name__) 12 | 13 | TOP_DIR = os.path.dirname(os.path.abspath(__file__)) 14 | RESOURCE_FILE = "susi.pmdl" 15 | 16 | 17 | class SnowboyDetector(HotwordDetector): 18 | """ 19 | This implements the Hotword Detector with Snowboy Hotword Detection Engine. 20 | """ 21 | 22 | def __init__(self, model = RESOURCE_FILE) -> None: 23 | super().__init__() 24 | self.detector = snowboydecoder.HotwordDetector( 25 | os.path.join(TOP_DIR, model), sensitivity=0.5) 26 | 27 | def run(self): 28 | """ 29 | Implementation of run abstract method in HotwordDetector. This method 30 | is called when thread is started for the first time. We start the 31 | Snowboy detection and declare detected callback as 32 | detection_callback method declared in parent class. 33 | """ 34 | pass 35 | #self.detector.start(detected_callback=self.on_detected, sleep_time=0.03) 36 | 37 | def start(self): 38 | logger.debug("SnowboyDetector: starting detection") 39 | self.detector.start(detected_callback=self.on_detected, sleep_time=0.03) 40 | 41 | def stop(self): 42 | logger.debug("SnowboyDetector: terminating detection") 43 | self.detector.terminate() 44 | -------------------------------------------------------------------------------- /susi_linux/hotword_engine/sphinx_detector.py: -------------------------------------------------------------------------------- 1 | """ 2 | Implementation of PocketSphinx Detector with PocketSphinx Speech Recognition 3 | Engine. It works on all the devices. 4 | """ 5 | import logging 6 | from pocketsphinx import LiveSpeech 7 | 8 | from .hotword_detector import HotwordDetector 9 | 10 | 11 | logger = logging.getLogger(__name__) 12 | 13 | 14 | class PocketSphinxDetector(HotwordDetector): 15 | """ 16 | This class implements Hotword Detection with the help of LiveSpeech Keyword 17 | spotting capabilities of PocketSphinx Speech Recognition Engine. 18 | """ 19 | 20 | def __init__(self) -> None: 21 | super().__init__() 22 | self.liveSpeech = LiveSpeech( 23 | lm=False, keyphrase='susi', kws_threshold=1e-20) 24 | 25 | def run(self): 26 | """ 27 | Implementation of run abstract method in HotwordDetector. This method is 28 | called when thread is started for the first time. We start the PocketSphinx 29 | LiveSpeech Keyword Spotting for detecting keyword 'susi' 30 | """ 31 | for phrase in self.liveSpeech: 32 | logger.info('Phrase: %s', phrase) 33 | if str(phrase) == 'susi': 34 | self.on_detected() 35 | -------------------------------------------------------------------------------- /susi_linux/hotword_engine/stop.pmdl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/susi_linux/hotword_engine/stop.pmdl -------------------------------------------------------------------------------- /susi_linux/hotword_engine/stopMusic.pmdl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/susi_linux/hotword_engine/stopMusic.pmdl -------------------------------------------------------------------------------- /susi_linux/hotword_engine/susi.pmdl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/susi_linux/hotword_engine/susi.pmdl -------------------------------------------------------------------------------- /susi_linux/internet_test.py: -------------------------------------------------------------------------------- 1 | """ 2 | Function to test the internet connection 3 | """ 4 | import logging 5 | import urllib.request 6 | from urllib.error import URLError 7 | 8 | 9 | logger = logging.getLogger(__name__) 10 | 11 | 12 | def internet_on(): 13 | url = 'http://216.58.192.142' 14 | try: 15 | # nosec #pylint-disable type: ignore 16 | urllib.request.urlopen(url, timeout=1) 17 | return True # pylint-enable 18 | except URLError as err: 19 | logger.error("Test %s failed. Error: %s", url, err) 20 | return False 21 | -------------------------------------------------------------------------------- /susi_linux/player.py: -------------------------------------------------------------------------------- 1 | """ Play via sound server """ 2 | 3 | import logging 4 | import requests 5 | from vlcplayer import vlcplayer 6 | 7 | logger = logging.getLogger(__name__) 8 | default_mode = 'server' 9 | baseurl = 'http://localhost:7070/' 10 | 11 | 12 | def send_request(req): 13 | try: 14 | requests.post(baseurl + req) 15 | except Exception as e: 16 | logger.error(e) 17 | 18 | 19 | class Player(): 20 | 21 | def __init__(self, mode=None): 22 | if mode is None: 23 | mode = default_mode 24 | if (not mode == 'server') and (not mode == 'direct'): 25 | logger.error('unknown mode %s, trying default mode', mode) 26 | mode = default_mode 27 | if mode == 'server': 28 | # try to check whether server is available 29 | try: 30 | requests.post(baseurl + 'status') 31 | self.mode = 'server' 32 | except Exception: 33 | self.mode = 'direct' 34 | logger.info('sound server not available, switching to direct play mode') 35 | else: 36 | self.mode = 'direct' 37 | logger.info('Player is working in mode: %s', self.mode) 38 | 39 | def _execute(self, method, mode=None): 40 | if (mode == 'server') or ((mode is None) and (self.mode == 'server')): 41 | send_request(method) 42 | else: 43 | getattr(vlcplayer, method)() 44 | 45 | def _executeArg(self, method, key, arg, mode=None): 46 | if (mode == 'server') or ((mode is None) and (self.mode == 'server')): 47 | send_request(method + '?' + key + '=' + arg) 48 | else: 49 | getattr(vlcplayer, method)(arg) 50 | 51 | def playytb(self, vid, mode=None): 52 | if (mode == 'server') or ((mode is None) and (self.mode == 'server')): 53 | send_request('play?ytb=' + vid) 54 | else: 55 | vlcplayer.playytb(vid) 56 | 57 | def play(self, mrl, mode=None): 58 | self._executeArg('play', 'mrl', mrl, mode) 59 | 60 | def pause(self, mode=None): 61 | self._execute('pause', mode) 62 | 63 | def resume(self, mode=None): 64 | self._execute('resume', mode) 65 | 66 | def next(self, mode=None): 67 | self._execute('next', mode) 68 | 69 | def previous(self, mode=None): 70 | self._execute('previous', mode) 71 | 72 | def restart(self, mode=None): 73 | self._execute('restart', mode) 74 | 75 | def stop(self, mode=None): 76 | self._execute('stop', mode) 77 | 78 | def beep(self, mrl, mode=None): 79 | self._executeArg('beep', 'mrl', mrl, mode) 80 | 81 | def say(self, mrl, mode=None): 82 | self._executeArg('say', 'mrl', mrl, mode) 83 | 84 | def shuffle(self, mode=None): 85 | self._execute('shuffle', mode) 86 | 87 | def volume(self, val, mode=None): 88 | self._executeArg('volume', 'val', val, mode) 89 | 90 | def save_softvolume(self, mode=None): 91 | self._execute('save_softvolume', mode) 92 | 93 | def restore_softvolume(self, mode=None): 94 | self._execute('restore_softvolume', mode) 95 | 96 | def save_hardvolume(self, mode=None): 97 | self._execute('save_hardvolume', mode) 98 | 99 | def restore_hardvolume(self, mode=None): 100 | self._execute('restore_hardvolume', mode) 101 | 102 | 103 | player = Player() 104 | -------------------------------------------------------------------------------- /susi_linux/speech/TTS.py: -------------------------------------------------------------------------------- 1 | """ This module implements all Text to Speech Services. 2 | You may use any of the speech synthesis services by calling the 3 | respective method. 4 | """ 5 | import logging 6 | import os 7 | import subprocess # nosec #pylint-disable type: ignore 8 | import tempfile 9 | from google_speech import Speech 10 | from watson_developer_cloud import TextToSpeechV1 11 | from ..player import player 12 | from susi_config import SusiConfig 13 | 14 | logger = logging.getLogger(__name__) 15 | susicfg = SusiConfig() 16 | 17 | text_to_speech = TextToSpeechV1( 18 | username=susicfg.get('watson.tts.user'), 19 | password=susicfg.get('watson.tts.pass')) 20 | 21 | 22 | def speak_flite_tts(text): 23 | """ This method implements Text to Speech using the Flite TTS. 24 | Flite TTS is completely offline. Usage of Flite is recommended if 25 | good internet connection is not available" 26 | :param text: Text which is needed to be spoken 27 | :return: None 28 | """ 29 | with tempfile.TemporaryDirectory() as tmpdirname: 30 | fd, filename = tempfile.mkstemp(text=True, dir=tmpdirname) 31 | with open(fd, 'w') as f: 32 | f.write(text) 33 | # Call flite tts to reply the response by Susi 34 | flite_speech_file = os.path.join( 35 | susicfg.get('path.base'), susicfg.get('path.flite_speech')) 36 | logger.debug( 37 | 'flite -voice file://%s -f %s', flite_speech_file, filename) 38 | fdout, wav_output = tempfile.mkstemp(suffix='.wav', dir=tmpdirname) 39 | subprocess.call( # nosec #pylint-disable type: ignore 40 | ['flite', '-v', '-voice', 'file://' + flite_speech_file, '-f', filename, '-o', wav_output]) # nosec #pylint-disable type: ignore 41 | player.say(wav_output) 42 | 43 | 44 | def speak_watson_tts(text): 45 | """ This method implements Text to Speech using the IBM Watson TTS. 46 | To use this, set username and password parameters in config file. 47 | :param text: Text which is needed to be spoken 48 | :return: None 49 | """ 50 | voice = susicfg.get('watson.tts.voice') 51 | if (voice == ''): 52 | lang = susicfg.get("language")[0:2] 53 | if lang == "en": 54 | voice = "en-US_AllisonVoice" 55 | elif lang == "de": 56 | voice = "de-DE_BirgitVoice" 57 | elif lang == "es": 58 | voice = "es-ES_LauraVoice" 59 | elif lang == "fr": 60 | voice = "fr-FR_ReneeVoice" 61 | elif lang == "it": 62 | voice = "it-IT_FrancescaVoice" 63 | elif lang == "ja": 64 | voice = "ja-JP_EmiVoice" 65 | elif lang == "pt": 66 | voice = "pt-BR_IsabelaVoice" 67 | else: 68 | # switch to English as default 69 | voice = "en-US_AllisonVoice" 70 | 71 | with tempfile.TemporaryDirectory() as tmpdirname: 72 | fd, wav_output = tempfile.mkstemp(suffix='.wav', dir=tmpdirname) 73 | with open(fd, 'wb') as audio_file: 74 | audio_file.write( 75 | text_to_speech.synthesize(text, accept='audio/wav', voice=voice)) 76 | 77 | player.say(wav_output) 78 | 79 | 80 | def speak_google_tts(text): 81 | """ This method implements Text to Speech using the Google Translate TTS. 82 | It uses Google Speech Python Package. 83 | :param text: Text which is needed to be spoken 84 | :return: None 85 | """ 86 | with tempfile.TemporaryDirectory() as tmpdirname: 87 | fd, mpiii = tempfile.mkstemp(suffix='.mp3', dir=tmpdirname) 88 | Speech(text=text, lang=susicfg.get("language")).save(mpiii) 89 | player.say(mpiii) 90 | 91 | # sox_effects = ("tempo", "1.2", "pitch", "2", "speed", "1") 92 | # player.save_softvolume() 93 | # player.volume(20) 94 | # Speech(text=text, lang='en').play(sox_effects) 95 | # player.restore_volume() 96 | -------------------------------------------------------------------------------- /susi_linux/speech/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/susi_linux/speech/__init__.py -------------------------------------------------------------------------------- /susi_linux/susi_loop.py: -------------------------------------------------------------------------------- 1 | """ 2 | Processing logic of susi_linux 3 | """ 4 | import time 5 | import os 6 | import re 7 | import logging 8 | import queue 9 | from threading import Thread, Timer, current_thread 10 | from datetime import datetime 11 | from urllib.parse import urljoin 12 | import speech_recognition as sr 13 | import requests 14 | import json_config 15 | import json 16 | import speech_recognition 17 | from speech_recognition import Recognizer, Microphone 18 | # from requests.exceptions import ConnectionError 19 | 20 | import susi_python as susi 21 | from .hardware_components.lights import lights 22 | from .internet_test import internet_on 23 | from .action_scheduler import ActionScheduler 24 | from .player import player 25 | from susi_config import SusiConfig 26 | from .speech import TTS 27 | 28 | logger = logging.getLogger(__name__) 29 | 30 | try: 31 | import RPi.GPIO as GPIO 32 | except ImportError: 33 | logger.warning("This device doesn't have GPIO port") 34 | GPIO = None 35 | 36 | class SusiLoop(): 37 | """The main SUSI loop dealing with hotword detection, voice recognition, 38 | server communication, action processing, etc""" 39 | 40 | def __init__(self, renderer=None): 41 | if GPIO: 42 | try: 43 | GPIO.setmode(GPIO.BCM) 44 | GPIO.setup(27, GPIO.OUT) 45 | GPIO.setup(22, GPIO.OUT) 46 | except RuntimeError as e: 47 | logger.error(e) 48 | 49 | thread1 = Thread(target=self.server_checker, name="ServerCheckerThread") 50 | thread1.daemon = True 51 | thread1.start() 52 | 53 | recognizer = Recognizer() 54 | # this was False in the old state machine, but reading the API docs 55 | # https://github.com/Uberi/speech_recognition/blob/master/reference/library-reference.rst 56 | # it seems that True is actually better! 57 | recognizer.dynamic_energy_threshold = True 58 | recognizer.energy_threshold = 2000 59 | self.recognizer = recognizer 60 | self.susi = susi 61 | self.renderer = renderer 62 | self.server_url = "https://127.0.0.1:4000" 63 | self.action_schduler = ActionScheduler() 64 | self.action_schduler.start() 65 | self.event_queue = queue.Queue() 66 | self.idle = True 67 | self.supported_languages = None 68 | 69 | try: 70 | res = requests.get('http://ip-api.com/json').json() 71 | self.susi.update_location( 72 | longitude=res['lon'], latitude=res['lat'], 73 | country_name=res['country'], country_code=res['countryCode']) 74 | 75 | except ConnectionError as e: 76 | logger.error(e) 77 | 78 | self.susi_config = SusiConfig() 79 | self.lang = self.susi_config.get('language') 80 | self.path_base = self.susi_config.get('path.base') 81 | self.sound_detection = os.path.abspath( 82 | os.path.join(self.path_base, 83 | self.susi_config.get('path.sound.detection'))) 84 | self.sound_problem = os.path.abspath( 85 | os.path.join(self.path_base, 86 | self.susi_config.get('path.sound.problem'))) 87 | self.sound_error_recognition = os.path.abspath( 88 | os.path.join(self.path_base, 89 | self.susi_config.get('path.sound.error.recognition'))) 90 | self.sound_error_timeout = os.path.abspath( 91 | os.path.join(self.path_base, 92 | self.susi_config.get('path.sound.error.timeout'))) 93 | 94 | if self.susi_config.get('susi.mode') == 'authenticated': 95 | try: 96 | susi.sign_in(email=self.susi_config.get('susi.user'), 97 | password=self.susi_config.get('susi.pass')) 98 | except Exception as e: 99 | logger.error('Some error occurred in login. Check you login details with susi-config.\n%s', e) 100 | 101 | if self.susi_config.get('hotword.engine') == 'Snowboy': 102 | from .hotword_engine.snowboy_detector import SnowboyDetector 103 | hotword_model = "susi.pmdl" 104 | if self.susi_config.get('hotword.model'): 105 | logger.debug("Using configured hotword model: " + self.susi_config.get('hotword.model')) 106 | hotword_model = self.susi_config.get('hotword_model') 107 | self.hotword_detector = SnowboyDetector(model=hotword_model) 108 | elif self.susi_config.get('hotword.engine') == 'PocketSphinx': 109 | from .hotword_engine.sphinx_detector import PocketSphinxDetector 110 | self.hotword_detector = PocketSphinxDetector() 111 | elif self.susi_config.get('hotword.engine') == 'None': 112 | self.hotword_detector = None 113 | else: 114 | raise ValueError(f"Unrecognized value for hotword.engine: {self.susi_config.get('hotword.engine')}") 115 | 116 | if self.susi_config.get('wakebutton') == 'enabled': 117 | logger.info("Susi has the wake button enabled") 118 | if self.susi_config.get('device') == 'RaspberryPi': 119 | logger.info("Susi runs on a RaspberryPi") 120 | from .hardware_components.rpi_wake_button import RaspberryPiWakeButton 121 | self.wake_button = RaspberryPiWakeButton() 122 | else: 123 | logger.warning("Susi is not running on a RaspberryPi") 124 | self.wake_button = None 125 | else: 126 | logger.warning("Susi has the wake button disabled") 127 | self.wake_button = None 128 | 129 | 130 | stt = self.susi_config.get('stt') 131 | if stt == 'google' or stt == 'watson' or stt == 'bing': 132 | # for internet based services we assume any language supported 133 | self.supported_languages = None 134 | elif stt == 'pocketsphinx': 135 | ps_data_dir = os.path.join(os.path.dirname(os.path.realpath(speech_recognition.__file__)), "pocketsphinx-data") 136 | self.supported_languages = [ f.name for f in os.scandir(ps_data_dir) if f.is_dir() ] 137 | logger.debug(f"Found supported languages for PocketSphinx: {self.supported_languages}") 138 | elif stt == 'deepspeech-local': 139 | ds_data_dir = os.path.join(os.path.dirname(os.path.realpath(speech_recognition.__file__)), "deepspeech-data") 140 | self.supported_languages = [ f.name for f in os.scandir(ds_data_dir) if f.is_dir() ] 141 | logger.debug(f"Found supported languages for DeepSpeech: {self.supported_languages}") 142 | elif stt == 'vosk': 143 | vosk_data_dir = os.path.join(os.path.dirname(os.path.realpath(speech_recognition.__file__)), "vosk-data") 144 | self.vosk_base_model_dir = vosk_data_dir 145 | self.supported_languages = [ f.name for f in os.scandir(vosk_data_dir) if f.is_dir() ] 146 | logger.debug(f"Found supported languages for Vosk: {self.supported_languages}") 147 | if (not self.lang in self.supported_languages): 148 | self.lang = "en" 149 | from vosk import Model 150 | self.vosk_model = Model(f"{vosk_data_dir}/{self.lang}") 151 | else: 152 | self.supported_languages = None 153 | logger.warn(f"Unknown stt setting: {stt}") 154 | 155 | if self.susi_config.get('stt') == 'deepspeech-local': 156 | self.microphone = Microphone(sample_rate=16000) 157 | else: 158 | self.microphone = Microphone() 159 | 160 | if self.hotword_detector is not None: 161 | self.hotword_detector.subject.subscribe( 162 | on_next=lambda x: self.hotword_detected_callback()) 163 | if self.wake_button is not None: 164 | self.wake_button.subject.subscribe( 165 | on_next=lambda x: self.hotword_detected_callback()) 166 | if self.renderer is not None: 167 | self.renderer.subject.subscribe( 168 | on_next=lambda x: self.hotword_detected_callback()) 169 | if self.action_schduler is not None: 170 | self.action_schduler.subject.subscribe( 171 | on_next=lambda x: self.queue_event(x)) 172 | 173 | def queue_event(self, event): 174 | """ queue a delayed event""" 175 | self.event_queue.put(event) 176 | 177 | def hotword_listener(self): 178 | """ thread function for listening to the hotword""" 179 | # this function never returns ... 180 | self.hotword_detector.start() 181 | 182 | def server_checker(self): 183 | """ thread function for checking the used server being alive""" 184 | response_one = None 185 | test_params = { 186 | 'q': 'Hello', 187 | 'timezoneOffset': int(time.timezone / 60) 188 | } 189 | while response_one is None: 190 | try: 191 | logger.debug("checking for local server") 192 | url = urljoin(self.server_url, '/susi/chat.json') 193 | response_one = requests.get(url, test_params).result() 194 | api_endpoint = self.server_url 195 | susi.use_api_endpoint(api_endpoint) 196 | except AttributeError: 197 | time.sleep(10) 198 | continue 199 | except ConnectionError: 200 | time.sleep(10) 201 | continue 202 | 203 | 204 | def start(self, background = False): 205 | """ start processing of audio events """ 206 | if self.hotword_detector is not None: 207 | hotword_thread = Thread(target=self.hotword_listener, name="HotwordDetectorThread") 208 | hotword_thread.daemon = True 209 | hotword_thread.start() 210 | 211 | if background: 212 | queue_loop_thread = Thread(target=self.queue_loop, name="QueueLoopThread") 213 | queue_loop_thread.daemon = True 214 | queue_loop_thread.start() 215 | else: 216 | self.queue_loop() 217 | 218 | 219 | def queue_loop(self): 220 | while True: 221 | # block until events are available 222 | ev = self.event_queue.get(block = True) 223 | logger.debug("Got event from event queue, trying to deal with it") 224 | # wait until idle 225 | while True: 226 | logger.debug("Waiting to become idle for planned action") 227 | if not self.idle: 228 | time.sleep(1) 229 | continue 230 | logger.debug("We are idle now ...") 231 | self.idle = False 232 | self.deal_with_answer(ev) 233 | # back from processing 234 | player.restore_softvolume() 235 | if GPIO: 236 | try: 237 | GPIO.output(27, False) 238 | GPIO.output(22, False) 239 | except RuntimeError: 240 | pass 241 | self.idle = True 242 | break 243 | 244 | 245 | def notify_renderer(self, message, payload=None): 246 | """ notify program renderer """ 247 | if self.renderer is not None: 248 | self.renderer.receive_message(message, payload) 249 | 250 | def hotword_detected_callback(self): 251 | """ 252 | Callback when the hotword is detected. Does the full processing 253 | logic formerly contained in different states 254 | """ 255 | logger.debug("Entering hotword callback") 256 | # don't do anything if we are already busy 257 | if not self.idle: 258 | logger.debug("Callback called while already busy, returning immediately from callback") 259 | return 260 | 261 | logger.debug("We are idle, so work on it!") 262 | self.idle = False 263 | 264 | # beep 265 | player.beep(self.sound_detection) 266 | 267 | if GPIO: 268 | GPIO.output(22, True) 269 | audio = None 270 | logger.debug("notify renderer for listening") 271 | self.notify_renderer('listening') 272 | with self.microphone as source: 273 | try: 274 | logger.debug("listening to voice command") 275 | audio = self.recognizer.listen(source, timeout=10.0, phrase_time_limit=5) 276 | except sr.WaitTimeoutError: 277 | logger.debug("timeout reached waiting for voice command") 278 | self.deal_with_error('ListenTimeout') 279 | logger.debug("delaying idle setting for 0.05s") 280 | Timer(interval=0.05, function=self.set_idle).start() 281 | return 282 | if GPIO: 283 | GPIO.output(22, False) 284 | 285 | lights.off() 286 | lights.think() 287 | try: 288 | logger.debug("Converting audio to text") 289 | value = self.recognize_audio(audio=audio, recognizer=self.recognizer) 290 | logger.debug("recognize_audio => %s", value) 291 | self.notify_renderer('recognized', value) 292 | if self.deal_with_answer(value): 293 | pass 294 | else: 295 | logger.error("Error dealing with answer") 296 | 297 | except sr.UnknownValueError as e: 298 | logger.error("UnknownValueError from SpeechRecognition: %s", e) 299 | self.deal_with_error('RecognitionError') 300 | 301 | logger.debug("delaying idle setting for 0.05s") 302 | Timer(interval=0.05, function=self.set_idle).start() 303 | return 304 | 305 | def set_idle(self): 306 | logger.debug("Switching to idle mode") 307 | self.notify_renderer('idle') 308 | self.idle = True 309 | 310 | def __speak(self, text): 311 | """Method to set the default TTS for the Speaker""" 312 | tts = self.susi_config.get('tts') 313 | if tts == 'google': 314 | TTS.speak_google_tts(text) 315 | elif tts == 'flite': 316 | logger.info("Using flite for TTS") # indication for using an offline music player 317 | TTS.speak_flite_tts(text) 318 | elif tts == 'watson': 319 | TTS.speak_watson_tts(text) 320 | else: 321 | raise ValueError("unknown key for tts", tts) 322 | 323 | def recognize_audio(self, recognizer, audio): 324 | """Use the configured STT method to convert spoken audio to text""" 325 | stt = self.susi_config.get('stt') 326 | lang = self.susi_config.get('language') 327 | # Try to adjust language to what is available 328 | # None indicates any language supported, so use it as is 329 | if self.supported_languages is not None: 330 | if len(self.supported_languages) == 0: 331 | raise ValueError(f"No supported language for the current STT {stt}") 332 | if "en-US" in self.supported_languages: 333 | default = "en-US" 334 | else: 335 | default = self.supported_languages[0] 336 | if lang not in self.supported_languages: 337 | if len(lang) < 2: 338 | logger.warn(f"Unsupported language code {lang}, using {default}") 339 | lang = default 340 | else: 341 | langshort = lang[0:2].lower() 342 | for l in self.supported_languages: 343 | if langshort == l[0:2].lower(): 344 | logger.debug(f"Using language code {l} instead of {lang}") 345 | lang = l 346 | break 347 | # We should now have a proper language code in lang, if not, warn and reset 348 | if lang not in self.supported_languages: 349 | logger.warn(f"Unsupported langauge code {lang}, using {default}") 350 | lang = default 351 | 352 | logger.info("Trying to recognize audio with %s in language: %s", stt, lang) 353 | if stt == 'google': 354 | return recognizer.recognize_google(audio, language=lang) 355 | 356 | elif stt == 'watson': 357 | username = self.susi_config.get('watson.stt.user') 358 | password = self.susi_config.get('watson.stt.pass') 359 | return recognizer.recognize_ibm( 360 | username=username, password=password, language=lang, audio_data=audio) 361 | 362 | elif stt == 'pocket_sphinx': 363 | return recognizer.recognize_sphinx(audio, language=lang) 364 | 365 | elif stt == 'bing': 366 | api_key = self.susi_config.get('bing.api') 367 | return recognizer.recognize_bing(audio_data=audio, key=api_key, language=lang) 368 | 369 | elif stt == 'deepspeech-local': 370 | return recognizer.recognize_deepspeech(audio, language=lang) 371 | 372 | elif stt == 'vosk': 373 | # TODO language support not implemented, we always use 374 | # the first language 375 | recognizer.vosk_model = self.vosk_model 376 | ret = json.loads(recognizer.recognize_vosk(audio, language=lang)) 377 | if ("text" in ret): 378 | return ret["text"] 379 | else: 380 | logger.error("Cannot detect text") 381 | return "" 382 | 383 | else: 384 | logger.error(f"Unknown STT setting: {stt}") 385 | logger.error("Using DeepSpeech!") 386 | return recognizer.recognize_deepspeech(audio, language=lang) 387 | 388 | 389 | 390 | def deal_with_error(self, payload=None): 391 | """deal with errors happening during processing of audio events""" 392 | if payload == 'RecognitionError': 393 | logger.debug("ErrorState Recognition Error") 394 | self.notify_renderer('error', 'recognition') 395 | lights.speak() 396 | player.say(self.sound_error_recognition) 397 | lights.off() 398 | elif payload == 'ConnectionError': 399 | self.notify_renderer('error', 'connection') 400 | self.susi_config.set('tts', 'flite') 401 | self.susi_config.set('stt', 'pocketsphinx') 402 | print("Internet Connection not available") 403 | lights.speak() 404 | lights.off() 405 | logger.info("Changed to offline providers") 406 | 407 | elif payload == 'ListenTimeout': 408 | self.notify_renderer('error', 'timeout') 409 | lights.speak() 410 | player.say(self.sound_error_timeout) 411 | lights.off() 412 | 413 | else: 414 | print("Error: {} \n".format(payload)) 415 | self.notify_renderer('error') 416 | lights.speak() 417 | player.say(self.sound_problem) 418 | lights.off() 419 | 420 | 421 | def deal_with_answer(self, payload=None): 422 | """processing logic - how to deal with answers from the server""" 423 | try: 424 | no_answer_needed = False 425 | 426 | if isinstance(payload, str): 427 | logger.debug("Sending payload to susi server: %s", payload) 428 | reply = self.susi.ask(payload) 429 | else: 430 | logger.debug("Executing planned action response: %s", payload) 431 | reply = payload 432 | 433 | if GPIO: 434 | GPIO.output(27, True) 435 | 436 | self.notify_renderer('speaking', payload={'susi_reply': reply}) 437 | 438 | if 'planned_actions' in reply.keys(): 439 | logger.debug("planning action: ") 440 | for plan in reply['planned_actions']: 441 | logger.debug("plan = " + str(plan)) 442 | # plan answers look like this: 443 | # plan = {'planned_actions': [{'language': 'en', 'answer': 'ALARM', 'plan_delay': 300001, 444 | # 'plan_date': '2020-01-09T02:05:10.377Z'}], 'language': 'en', 'answer': 'alarm set for in 5 minutes'} 445 | # we use time.time as timefunc for scheduler, so we need to convert the 446 | # delay and absolute time to the same format, that is float of sec since epoch 447 | # Unfortunately, Python is tooooooo stupid to provide ISO standard confirm standard 448 | # library. datetime.fromisoformat sounds like perfectly made, only that it doesn't 449 | # parse the Z postfix, congratulations. 450 | # https://discuss.python.org/t/parse-z-timezone-suffix-in-datetime/2220 451 | # Replace it manually with +00:00 452 | # We send both the delay and absolute time in case one of the two is missing 453 | # the scheduler prefers the delay value 454 | plan_date_sec = datetime.fromisoformat(re.sub('Z$', '+00:00', plan['plan_date'])).timestamp() 455 | self.action_schduler.add_event(int(plan['plan_delay']) / 1000, plan_date_sec, plan) 456 | 457 | # first responses WITHOUT answer key! 458 | 459 | # {'answer': 'Audio volume is now 10 percent.', 'volume': '10'} 460 | if 'volume' in reply.keys(): 461 | no_answer_needed = True 462 | player.volume(reply['volume']) 463 | player.say(self.sound_detection) 464 | 465 | if 'media_action' in reply.keys(): 466 | action = reply['media_action'] 467 | if action == 'pause': 468 | no_answer_needed = True 469 | player.pause() 470 | lights.off() 471 | lights.wakeup() 472 | elif action == 'resume': 473 | no_answer_needed = True 474 | player.resume() 475 | elif action == 'restart': 476 | no_answer_needed = True 477 | player.restart() 478 | elif action == 'next': 479 | no_answer_needed = True 480 | player.next() 481 | elif action == 'previous': 482 | no_answer_needed = True 483 | player.previous() 484 | elif action == 'shuffle': 485 | no_answer_needed = True 486 | player.shuffle() 487 | else: 488 | logger.error('Unknown media action: %s', action) 489 | 490 | # {'stop': } 491 | if 'stop' in reply.keys(): 492 | no_answer_needed = True 493 | player.stop() 494 | 495 | if 'answer' in reply.keys(): 496 | logger.info('Susi: %s', reply['answer']) 497 | lights.off() 498 | lights.speak() 499 | self.__speak(reply['answer']) 500 | lights.off() 501 | else: 502 | if not no_answer_needed and 'identifier' not in reply.keys(): 503 | lights.off() 504 | lights.speak() 505 | self.__speak("I don't have an answer to this") 506 | lights.off() 507 | 508 | if 'language' in reply.keys(): 509 | answer_lang = reply['language'] 510 | if answer_lang != self.susi_config.get("language"): 511 | logger.info("Switching language to: %s", answer_lang) 512 | # switch language 513 | self.susi_config.set('language', answer_lang) 514 | # TODO 515 | # for vosk we need to update self.vosk_model = Model(f"{self.vosk_model_base}/{answer_lang}") 516 | # given that the language is supported! 517 | 518 | # answer to "play ..." 519 | # {'identifier': 'ytd-04854XqcfCY', 'answer': 'Playing Queen - We Are The Champions (Official Video)'} 520 | if 'identifier' in reply.keys(): 521 | url = reply['identifier'] 522 | logger.debug("Playing " + url) 523 | if url[:3] == 'ytd': 524 | player.playytb(url[4:]) 525 | else: 526 | player.play(url) 527 | 528 | if 'table' in reply.keys(): 529 | table = reply['table'] 530 | for h in table.head: 531 | print('%s\t' % h, end='') 532 | self.__speak(h) 533 | print() 534 | for datum in table.data[0:4]: 535 | for value in datum: 536 | print('%s\t' % value, end='') 537 | self.__speak(value) 538 | print() 539 | 540 | if 'rss' in reply.keys(): 541 | rss = reply['rss'] 542 | entities = rss['entities'] 543 | count = rss['count'] 544 | for entity in entities[0:count]: 545 | logger.debug(entity.title) 546 | self.__speak(entity.title) 547 | 548 | except ConnectionError: 549 | self.deal_with_error('ConnectionError') 550 | return False 551 | except Exception as e: 552 | logger.error('Unknown error: %s', e) 553 | return False 554 | 555 | return True 556 | 557 | -------------------------------------------------------------------------------- /susi_linux/ui/__init__.py: -------------------------------------------------------------------------------- 1 | from .login_window import LoginWindow 2 | from .configuration_window import ConfigurationWindow 3 | from .app_window import SusiAppWindow 4 | -------------------------------------------------------------------------------- /susi_linux/ui/animators.py: -------------------------------------------------------------------------------- 1 | import math 2 | import cairo 3 | import gi 4 | gi.require_version('Gtk', '3.0') 5 | from gi.repository import Gtk, GLib # nopep8 6 | 7 | 8 | class Animator(Gtk.DrawingArea): 9 | def __init__(self, **properties): 10 | super().__init__(**properties) 11 | self.set_size_request(200, 80) 12 | self.connect("draw", self.do_drawing) 13 | GLib.timeout_add(50, self.tick) 14 | 15 | def tick(self): 16 | self.queue_draw() 17 | return True 18 | 19 | def do_drawing(self, widget, cr): 20 | self.draw(cr, self.get_allocated_width(), self.get_allocated_height()) 21 | 22 | def draw(self, ctx, width, height): 23 | pass 24 | 25 | 26 | class ListeningAnimator(Animator): 27 | def __init__(self, window, **properties): 28 | super().__init__(**properties) 29 | self.window = window 30 | self.tc = 0 31 | 32 | def draw(self, ctx, width, height): 33 | 34 | self.tc += 0.2 35 | self.tc %= 2 * math.pi 36 | 37 | for i in range(-4, 5): 38 | ctx.set_source_rgb(0.2, 0.5, 1) 39 | ctx.set_line_width(6) 40 | ctx.set_line_cap(cairo.LINE_CAP_ROUND) 41 | if i % 2 == 0: 42 | ctx.move_to(width / 2 + i * 10, height / 2 + 3 - 8 * math.sin(self.tc + i)) 43 | ctx.line_to(width / 2 + i * 10, height / 2 - 3 + 8 * math.sin(self.tc + i)) 44 | else: 45 | ctx.set_source_rgb(0.2, 0.7, 1) 46 | ctx.move_to(width / 2 + i * 10, height / 2 + 3 - 8 * math.cos(self.tc - i)) 47 | ctx.line_to(width / 2 + i * 10, height / 2 - 3 + 8 * math.cos(self.tc - i)) 48 | ctx.stroke() 49 | 50 | 51 | class ThinkingAnimator(Animator): 52 | def __init__(self, window, **properties): 53 | super().__init__(**properties) 54 | self.window = window 55 | self.rot = 0 56 | self.x, self.y = 0, 0 57 | self.rad = 20 58 | 59 | def draw(self, ctx, width, height): 60 | self.x, self.y = width / 2, height / 2 61 | self.rot += 0.2 62 | self.rot %= 2 * math.pi 63 | 64 | for i in range(-2, 2): 65 | ctx.set_source_rgb(0.2, 0.7, 1) 66 | ctx.arc( 67 | self.x + i * 20, 68 | self.y, 69 | 8 * math.cos(self.rot - i / 2), 70 | 0, 71 | 2 * math.pi) 72 | ctx.fill() 73 | -------------------------------------------------------------------------------- /susi_linux/ui/app_window.py: -------------------------------------------------------------------------------- 1 | import os 2 | import signal 3 | import logging 4 | import gi 5 | from . import ConfigurationWindow 6 | gi.require_version('Gtk', '3.0') # nopep8 7 | from async_promises import Promise 8 | from .animators import ListeningAnimator, ThinkingAnimator 9 | from .renderer import Renderer 10 | from gi.repository import Gtk 11 | 12 | TOP_DIR = os.path.dirname(os.path.abspath(__file__)) 13 | logger = logging.getLogger(__name__) 14 | 15 | 16 | class SusiAppWindow(Renderer): 17 | def __init__(self): 18 | super().__init__() 19 | builder = Gtk.Builder() 20 | builder.add_from_file( 21 | os.path.join(TOP_DIR, "glade_files/susi_app.glade")) 22 | 23 | self.window = builder.get_object("app_window") 24 | self.user_text_label = builder.get_object("user_text_label") 25 | self.susi_text_label = builder.get_object("susi_text_label") 26 | self.root_box = builder.get_object("root_box") 27 | self.state_stack = builder.get_object("state_stack") 28 | self.mic_button = builder.get_object("mic_button") 29 | self.mic_box = builder.get_object("mic_box") 30 | self.listening_box = builder.get_object("listening_box") 31 | self.thinking_box = builder.get_object("thinking_box") 32 | self.error_label = builder.get_object("error_label") 33 | self.settings_button = builder.get_object("settings_button") 34 | 35 | listeningAnimator = ListeningAnimator(self.window) 36 | self.listening_box.add(listeningAnimator) 37 | self.listening_box.reorder_child(listeningAnimator, 1) 38 | self.listening_box.set_child_packing( 39 | listeningAnimator, False, False, 0, Gtk.PackType.END) 40 | 41 | thinkingAnimator = ThinkingAnimator(self.window) 42 | self.thinking_box.add(thinkingAnimator) 43 | self.thinking_box.reorder_child(thinkingAnimator, 1) 44 | self.thinking_box.set_child_packing( 45 | thinkingAnimator, False, False, 0, Gtk.PackType.END) 46 | 47 | builder.connect_signals(SusiAppWindow.Handler(self)) 48 | self.window.set_default_size(300, 600) 49 | self.window.set_resizable(False) 50 | 51 | def show_window(self): 52 | self.window.show_all() 53 | Gtk.main() 54 | 55 | def exit_window(self): 56 | self.window.destroy() 57 | Gtk.main_quit() 58 | 59 | def receive_message(self, message_type, payload=None): 60 | if message_type == 'idle': 61 | self.state_stack.set_visible_child_name("mic_page") 62 | 63 | elif message_type == 'listening': 64 | self.state_stack.set_visible_child_name("listening_page") 65 | self.user_text_label.set_text("") 66 | self.susi_text_label.set_text("") 67 | 68 | elif message_type == 'recognizing': 69 | self.state_stack.set_visible_child_name("thinking_page") 70 | 71 | elif message_type == 'recognized': 72 | user_text = payload 73 | self.user_text_label.set_text(user_text) 74 | 75 | elif message_type == 'speaking': 76 | self.state_stack.set_visible_child_name("empty_page") 77 | susi_reply = payload['susi_reply'] 78 | if 'answer' in susi_reply.keys(): 79 | self.susi_text_label.set_text(susi_reply['answer']) 80 | 81 | elif message_type == 'error': 82 | self.state_stack.set_visible_child_name("error_page") 83 | error_type = payload 84 | if error_type is not None: 85 | if error_type == 'connection': 86 | self.error_label.set_text( 87 | "Problem in internet connectivity !!") 88 | elif error_type == 'recognition': 89 | self.error_label.set_text("Couldn't recognize the speech.") 90 | else: 91 | self.error_label.set_text('Some error occurred,') 92 | 93 | class Handler: 94 | def __init__(self, app_window): 95 | self.app_window = app_window 96 | 97 | def on_delete(self, *args): 98 | self.app_window.exit_window() 99 | os.kill(os.getpid(), signal.SIGHUP) 100 | 101 | def on_mic_button_clicked(self, button): 102 | Promise( 103 | lambda resolve, reject: resolve(self.app_window.on_mic_pressed()) 104 | ) 105 | 106 | def on_settings_button_clicked(self, button): 107 | window = ConfigurationWindow() 108 | window.show_window() 109 | -------------------------------------------------------------------------------- /susi_linux/ui/configuration_window.py: -------------------------------------------------------------------------------- 1 | import os 2 | import logging 3 | import gi 4 | gi.require_version('Gtk', '3.0') # nopep8 5 | from pathlib import Path 6 | from gi.repository import Gtk 7 | from .login_window import LoginWindow 8 | from susi_config import SusiConfig 9 | 10 | TOP_DIR = os.path.dirname(os.path.abspath(__file__)) 11 | susicfg = SusiConfig() 12 | 13 | # 14 | # WARNING!!!! 15 | # This order needs to be synced with glade_files/configure.glade 16 | # which defines the actual strings and entries in the list!!!! 17 | STT_DEEPSPEECH=0 18 | STT_VOSK=1 19 | STT_GOOGLE=2 20 | STT_WATSON=3 21 | STT_BING=4 22 | 23 | TTS_FLITE=0 24 | TTS_GOOGLE=1 25 | TTS_WATSON=2 26 | 27 | HOTWORD_SNOWBOY=0 28 | HOTWORD_POCKETSPHINX=1 29 | HOTWORD_NONE=2 30 | 31 | class WatsonCredentialsDialog(Gtk.Dialog): 32 | def __init__(self, parent): 33 | Gtk.Dialog.__init__(self, "Enter Credentials", parent, 0, 34 | (Gtk.STOCK_CANCEL, Gtk.ResponseType.CANCEL, 35 | Gtk.STOCK_OK, Gtk.ResponseType.OK)) 36 | 37 | self.set_default_size(150, 100) 38 | 39 | username_field = Gtk.Entry() 40 | username_field.set_placeholder_text("Username") 41 | password_field = Gtk.Entry() 42 | password_field.set_placeholder_text("Password") 43 | password_field.set_visibility(False) 44 | password_field.set_invisible_char('*') 45 | 46 | self.username_field = username_field 47 | self.password_field = password_field 48 | 49 | box = self.get_content_area() 50 | 51 | box.set_margin_top(10) 52 | box.set_margin_bottom(10) 53 | box.set_margin_left(10) 54 | box.set_margin_right(10) 55 | 56 | box.set_spacing(10) 57 | 58 | box.add(username_field) 59 | box.add(password_field) 60 | self.show_all() 61 | 62 | 63 | class BingCredentialDialog(Gtk.Dialog): 64 | def __init__(self, parent): 65 | Gtk.Dialog.__init__(self, "Enter API Key", parent, 0, 66 | (Gtk.STOCK_CANCEL, Gtk.ResponseType.CANCEL, 67 | Gtk.STOCK_OK, Gtk.ResponseType.OK)) 68 | 69 | self.set_default_size(150, 100) 70 | 71 | api_key_field = Gtk.Entry() 72 | api_key_field.set_placeholder_text("API Key") 73 | 74 | self.api_key_field = api_key_field 75 | 76 | box = self.get_content_area() 77 | 78 | box.set_margin_top(10) 79 | box.set_margin_bottom(10) 80 | box.set_margin_left(10) 81 | box.set_margin_right(10) 82 | 83 | box.set_spacing(10) 84 | 85 | box.add(api_key_field) 86 | self.show_all() 87 | 88 | 89 | class ConfigurationWindow: 90 | def __init__(self) -> None: 91 | super().__init__() 92 | builder = Gtk.Builder() 93 | builder.add_from_file(os.path.join( 94 | TOP_DIR, "glade_files/configure.glade")) 95 | 96 | self.window = builder.get_object("configuration_window") 97 | self.stt_combobox = builder.get_object("stt_combobox") 98 | self.tts_combobox = builder.get_object("tts_combobox") 99 | self.auth_switch = builder.get_object("auth_switch") 100 | self.hotword_combobox = builder.get_object("hotword_combobox") 101 | self.wake_button_switch = builder.get_object("wake_button_switch") 102 | 103 | self.init_auth_switch() 104 | self.init_tts_combobox() 105 | self.init_stt_combobox() 106 | self.init_hotword_switch() 107 | self.init_wake_button_switch() 108 | 109 | builder.connect_signals(ConfigurationWindow.Handler(self)) 110 | self.window.set_resizable(False) 111 | 112 | def show_window(self): 113 | self.window.show_all() 114 | Gtk.main() 115 | 116 | def exit_window(self): 117 | self.window.destroy() 118 | Gtk.main_quit() 119 | 120 | def init_tts_combobox(self): 121 | default_tts = susicfg.get('tts') 122 | if default_tts == 'google': 123 | self.tts_combobox.set_active(TTS_GOOGLE) 124 | elif default_tts == 'flite': 125 | self.tts_combobox.set_active(TTS_FLITE) 126 | elif default_tts == 'watson': 127 | self.tts_combobox.set_active(TTS_WATSON) 128 | else: 129 | self.tts_combobox.set_active(TTS_FLITE) 130 | susicfg.set('tts', 'flite') 131 | 132 | def init_stt_combobox(self): 133 | default_stt = susicfg.get('stt') 134 | if default_stt == 'google': 135 | self.stt_combobox.set_active(STT_GOOGLE) 136 | elif default_stt == 'watson': 137 | self.stt_combobox.set_active(STT_WATSON) 138 | elif default_stt == 'bing': 139 | self.stt_combobox.set_active(STT_BING) 140 | elif default_stt == 'deepspeech-local': 141 | self.stt_combobox.set_active(STT_DEEPSPEECH) 142 | elif default_stt == 'vosk': 143 | self.stt_combobox.set_active(STT_VOSK) 144 | else: 145 | self.tts_combobox.set_active(STT_DEEPSPEECH) 146 | susicfg.set('stt', 'deepspeech-local') 147 | 148 | def init_auth_switch(self): 149 | usage_mode = susicfg.get('susi.mode') 150 | if usage_mode == 'authenticated': 151 | self.auth_switch.set_active(True) 152 | else: 153 | self.auth_switch.set_active(False) 154 | 155 | def init_hotword_switch(self): 156 | default_hotword = susicfg.get('hotword.engine') 157 | if default_hotword == 'Snowboy': 158 | self.hotword_combobox.set_active(HOTWORD_SNOWBOY) 159 | elif default_hotword == 'PocketSphinx': 160 | self.hotword_combobox.set_active(HOTWORD_POCKETSPHINX) 161 | elif default_hotword == 'None': 162 | self.hotword_combobox.set_active(HOTWORD_NONE) 163 | else: 164 | try: 165 | import snowboy 166 | self.hotword_combobox.set_active(HOTWORD_SNOWBOY) 167 | susicfg.set('hotword.engine', 'Snowboy') 168 | except ImportError: 169 | self.hotword_combobox.set_active(HOTWORD_POCKETSPHINX) 170 | susicfg.set('hotword.engine', 'PocketSphinx') 171 | 172 | def init_wake_button_switch(self): 173 | try: 174 | import RPi.GPIO 175 | if susicfg.get('wakebutton') == 'enabled': 176 | self.wake_button_switch.set_active(True) 177 | else: 178 | self.wake_button_switch.set_active(False) 179 | except ImportError: 180 | self.wake_button_switch.set_sensitive(False) 181 | except RuntimeError: 182 | self.wake_button_switch.set_sensitive(False) 183 | 184 | class Handler: 185 | def __init__(self, config_window): 186 | self.config_window = config_window 187 | 188 | def on_delete_window(self, *args): 189 | self.config_window.exit_window() 190 | 191 | def on_stt_combobox_changed(self, combo: Gtk.ComboBox): 192 | selection = combo.get_active() 193 | 194 | if selection == STT_DEEPSPEECH: 195 | susicfg.set('stt', 'deepspeech_local') 196 | 197 | elif selection == STT_VOSK: 198 | susicfg.set('stt', 'vosk') 199 | 200 | elif selection == STT_GOOGLE: 201 | susicfg.set('stt', 'google') 202 | 203 | elif selection == STT_WATSON: 204 | credential_dialog = WatsonCredentialsDialog( 205 | self.config_window.window) 206 | response = credential_dialog.run() 207 | 208 | if response == Gtk.ResponseType.OK: 209 | username = credential_dialog.username_field.get_text() 210 | password = credential_dialog.password_field.get_text() 211 | susicfg.set('stt', 'watson') 212 | susicfg.set('watson.stt.user', username) 213 | susicfg.set('watson.stt.pass', password) 214 | else: 215 | self.config_window.init_stt_combobox() 216 | 217 | credential_dialog.destroy() 218 | 219 | elif selection == STT_BING: 220 | credential_dialog = BingCredentialDialog( 221 | self.config_window.window) 222 | response = credential_dialog.run() 223 | 224 | if response == Gtk.ResponseType.OK: 225 | api_key = credential_dialog.api_key_field.get_text() 226 | susicfg.set('stt', 'bing') 227 | susicfg.set('bing.api', api_key) 228 | else: 229 | self.config_window.init_stt_combobox() 230 | 231 | credential_dialog.destroy() 232 | 233 | def on_tts_combobox_changed(self, combo): 234 | selection = combo.get_active() 235 | 236 | if selection == TTS_GOOGLE: 237 | susicfg.set('tts', 'google') 238 | 239 | elif selection == TTS_FLITE: 240 | susicfg.set('tts', 'flite') 241 | 242 | elif selection == TTS_WATSON: 243 | credential_dialog = WatsonCredentialsDialog( 244 | self.config_window.window) 245 | response = credential_dialog.run() 246 | 247 | if response == Gtk.ResponseType.OK: 248 | username = credential_dialog.username_field.get_text() 249 | password = credential_dialog.password_field.get_text() 250 | susicfg.set('tts', 'watson') 251 | susicfg.set('watson.tts.user', username) 252 | susicfg.set('watson.tts.pass', password) 253 | susicfg.set('watson.tts.voice', 'en-US_AllisonVoice') 254 | else: 255 | self.config_window.init_tts_combobox() 256 | credential_dialog.destroy() 257 | 258 | def on_auth_switch_active_notify(self, switch, gparam): 259 | if switch.get_active(): 260 | login_window = LoginWindow() 261 | login_window.show_window() 262 | if susicfg.get('susi.mode') == 'authenticated': 263 | switch.set_active(True) 264 | else: 265 | switch.set_active(False) 266 | 267 | def on_hotword_combobox_changed(self, combo: Gtk.ComboBox): 268 | selection = combo.get_active() 269 | 270 | if selection == HOTWORD_SNOWBOY: 271 | susicfg.set('hotword.engine', 'Snowboy') 272 | elif selection == HOTWORD_POCKETSPHINX: 273 | susicfg.set('hotword.engine', 'PocketSphinx') 274 | elif selection == HOTWORD_NONE: 275 | susicfg.set('hotword.engine', 'None') 276 | 277 | 278 | def on_wake_button_switch_active_notify(self, switch, gparam): 279 | if switch.get_active(): 280 | susicfg.set('wakebutton', 'enabled') 281 | else: 282 | susicfg.set('wakebutton', 'disabled') 283 | -------------------------------------------------------------------------------- /susi_linux/ui/glade_files/configure.glade: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | Deepspeech 14 | 15 | 16 | Vosk 17 | 18 | 19 | Google 20 | 21 | 22 | IBM Watson 23 | 24 | 25 | Bing 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | Flite TTS 37 | 38 | 39 | Google Translate TTS 40 | 41 | 42 | IBM Watson 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | Snowboy 54 | 55 | 56 | PocketSphinx 57 | 58 | 59 | None 60 | 61 | 62 | 63 | 64 | False 65 | Configure Settings for SUSI Linux 66 | 67 | 68 | 69 | True 70 | False 71 | 10 72 | 10 73 | 10 74 | 10 75 | vertical 76 | 8 77 | 78 | 79 | 150 80 | True 81 | False 82 | images/susi_icon.png 83 | 84 | 85 | False 86 | True 87 | 0 88 | 89 | 90 | 91 | 92 | True 93 | False 94 | 10 95 | 10 96 | 20 97 | 98 | 99 | True 100 | False 101 | 10 102 | 10 103 | Default Speech Recognition Service 104 | 105 | 106 | False 107 | True 108 | 0 109 | 110 | 111 | 112 | 113 | 200 114 | True 115 | False 116 | stt_liststore 117 | 0 118 | 119 | 120 | 121 | 122 | 0 123 | 124 | 125 | 126 | 127 | False 128 | True 129 | end 130 | 1 131 | 132 | 133 | 134 | 135 | False 136 | True 137 | 1 138 | 139 | 140 | 141 | 142 | True 143 | False 144 | 10 145 | 10 146 | 20 147 | 148 | 149 | True 150 | False 151 | 10 152 | 10 153 | Default Text to Speech Engine 154 | 155 | 156 | False 157 | True 158 | 0 159 | 160 | 161 | 162 | 163 | 200 164 | True 165 | False 166 | tts_liststore 167 | 0 168 | 1 169 | 170 | 171 | 172 | 173 | 0 174 | 175 | 176 | 177 | 178 | False 179 | True 180 | end 181 | 1 182 | 183 | 184 | 185 | 186 | False 187 | True 188 | 2 189 | 190 | 191 | 192 | 193 | True 194 | False 195 | 20 196 | top 197 | 198 | 199 | True 200 | False 201 | 10 202 | 10 203 | 10 204 | Use Authenticated Mode 205 | 206 | 207 | False 208 | True 209 | 0 210 | 211 | 212 | 213 | 214 | True 215 | True 216 | 10 217 | 218 | 219 | 220 | False 221 | True 222 | end 223 | 1 224 | 225 | 226 | 227 | 228 | False 229 | True 230 | 3 231 | 232 | 233 | 234 | 235 | True 236 | False 237 | 10 238 | 10 239 | 20 240 | 241 | 242 | True 243 | False 244 | 10 245 | 10 246 | Hotword Detection System 247 | 248 | 249 | False 250 | True 251 | 0 252 | 253 | 254 | 255 | 256 | 200 257 | True 258 | False 259 | hotword_liststore 260 | 0 261 | 1 262 | 263 | 264 | 265 | 266 | 0 267 | 268 | 269 | 270 | 271 | False 272 | True 273 | end 274 | 1 275 | 276 | 277 | 278 | 279 | False 280 | True 281 | 4 282 | 283 | 284 | 285 | 286 | True 287 | False 288 | 10 289 | 10 290 | 20 291 | 292 | 293 | True 294 | False 295 | 10 296 | 10 297 | Use Wake Button 298 | 299 | 300 | False 301 | True 302 | 0 303 | 304 | 305 | 306 | 307 | True 308 | True 309 | 310 | 311 | 312 | False 313 | True 314 | end 315 | 1 316 | 317 | 318 | 319 | 320 | False 321 | True 322 | 5 323 | 324 | 325 | 326 | 327 | 328 | 329 | -------------------------------------------------------------------------------- /susi_linux/ui/glade_files/images/error.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/susi_linux/ui/glade_files/images/error.png -------------------------------------------------------------------------------- /susi_linux/ui/glade_files/images/microphone.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/susi_linux/ui/glade_files/images/microphone.png -------------------------------------------------------------------------------- /susi_linux/ui/glade_files/images/susi_icon.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/susi_linux/ui/glade_files/images/susi_icon.png -------------------------------------------------------------------------------- /susi_linux/ui/glade_files/signin.glade: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | False 8 | Login to SUSI.AI 9 | 10 | 11 | 12 | True 13 | False 14 | 10 15 | 10 16 | 10 17 | 10 18 | vertical 19 | 10 20 | 21 | 22 | 400 23 | 100 24 | True 25 | False 26 | 50 27 | images/susi_icon.png 28 | 29 | 30 | False 31 | True 32 | 0 33 | 34 | 35 | 36 | 37 | True 38 | True 39 | 40 40 | 40 41 | Email ID 42 | email 43 | 44 | 45 | 46 | True 47 | True 48 | 1 49 | 50 | 51 | 52 | 53 | True 54 | True 55 | 40 56 | 40 57 | False 58 | * 59 | Password 60 | password 61 | 62 | 63 | 64 | True 65 | True 66 | 2 67 | 68 | 69 | 70 | 71 | 50 72 | True 73 | False 74 | 6 75 | 6 76 | 77 | 78 | False 79 | True 80 | 3 81 | 82 | 83 | 84 | 85 | Sign In 86 | True 87 | True 88 | True 89 | 40 90 | 40 91 | 30 92 | 20 93 | 0 94 | 95 | 96 | 97 | True 98 | True 99 | 4 100 | 101 | 102 | 103 | 104 | 105 | 106 | -------------------------------------------------------------------------------- /susi_linux/ui/glade_files/susi_app.glade: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | True 8 | False 9 | images/microphone.png 10 | 11 | 12 | False 13 | False 14 | 15 | 16 | 17 | True 18 | False 19 | 20 | 21 | True 22 | False 23 | 10 24 | 10 25 | vertical 26 | 10 27 | 28 | 29 | True 30 | False 31 | end 32 | right 33 | True 34 | 60 35 | 60 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | False 46 | True 47 | 1 48 | 49 | 50 | 51 | 52 | True 53 | False 54 | True 55 | 60 56 | 60 57 | 58 | 59 | 60 | 61 | 62 | 63 | False 64 | True 65 | 2 66 | 67 | 68 | 69 | 70 | True 71 | False 72 | slide-up-down 73 | 74 | 75 | True 76 | False 77 | 78 | 79 | True 80 | False 81 | Say 'SUSI' or press the mic button 82 | 83 | 84 | 85 | 86 | 87 | True 88 | True 89 | 0 90 | 91 | 92 | 93 | 94 | True 95 | True 96 | True 97 | mic_icon 98 | none 99 | 100 | 101 | 102 | False 103 | False 104 | end 105 | 1 106 | 107 | 108 | 109 | 110 | 111 | 112 | 113 | mic_page 114 | True 115 | 116 | 117 | 118 | 119 | True 120 | False 121 | 122 | 123 | True 124 | False 125 | Speak Now 126 | 127 | 128 | 129 | 130 | 131 | True 132 | True 133 | 0 134 | 135 | 136 | 137 | 138 | 139 | 140 | 141 | listening_page 142 | 1 143 | True 144 | 145 | 146 | 147 | 148 | True 149 | False 150 | 151 | 152 | True 153 | False 154 | Thinking 155 | 156 | 157 | 158 | 159 | 160 | True 161 | True 162 | 0 163 | 164 | 165 | 166 | 167 | 168 | 169 | 170 | thinking_page 171 | page0 172 | 2 173 | 174 | 175 | 176 | 177 | True 178 | False 179 | vertical 180 | 181 | 182 | 183 | 184 | 185 | empty_page 186 | page0 187 | 3 188 | 189 | 190 | 191 | 192 | True 193 | False 194 | 195 | 196 | True 197 | False 198 | An Error Occurred 199 | 200 | 201 | 202 | 203 | 204 | True 205 | True 206 | 0 207 | 208 | 209 | 210 | 211 | True 212 | False 213 | images/error.png 214 | 215 | 216 | False 217 | True 218 | 1 219 | 220 | 221 | 222 | 223 | error_page 224 | page0 225 | 4 226 | 227 | 228 | 229 | 230 | False 231 | True 232 | end 233 | 4 234 | 235 | 236 | 237 | 238 | -1 239 | 240 | 241 | 242 | 243 | True 244 | False 245 | start 246 | SUSI 247 | 248 | 249 | gtk-preferences 250 | True 251 | True 252 | False 253 | True 254 | True 255 | 256 | 257 | 258 | 259 | 260 | 261 | 262 | 263 | 264 | -------------------------------------------------------------------------------- /susi_linux/ui/login_window.py: -------------------------------------------------------------------------------- 1 | import os 2 | import re 3 | import gi 4 | import requests 5 | import json_config 6 | gi.require_version('Gtk', '3.0') # nopep8 7 | from gi.repository import Gtk 8 | from gi.repository.Gdk import Color 9 | 10 | TOP_DIR = os.path.dirname(os.path.abspath(__file__)) 11 | config = json_config.connect('config.json') 12 | 13 | 14 | def is_valid(email, password): 15 | """ 16 | Method to Validate SUSI Login Details 17 | :param email: SUSI Sign-in email 18 | :param password: SUSI Sign-in password 19 | :return: boolean to indicate if details are valid 20 | """ 21 | params = { 22 | 'login': email, 23 | 'password': password 24 | } 25 | sign_in_url = 'http://api.susi.ai/aaa/login.json?type=access-token' 26 | api_response = requests.get(sign_in_url, params) 27 | # except OSError: 28 | # raise ConnectionError 29 | 30 | if api_response.status_code == 200: 31 | return True 32 | else: 33 | return False 34 | 35 | 36 | class LoginWindow(): 37 | def __init__(self): 38 | builder = Gtk.Builder() 39 | builder.add_from_file(os.path.join( 40 | TOP_DIR, "glade_files/signin.glade")) 41 | 42 | self.window = builder.get_object("login_window") 43 | self.email_field = builder.get_object("email_field") 44 | self.password_field = builder.get_object("password_field") 45 | self.spinner = builder.get_object("signin_spinner") 46 | self.sign_in_button = builder.get_object("signin_button") 47 | self.sign_in_button.set_sensitive(False) 48 | 49 | builder.connect_signals(LoginWindow.Handler(self)) 50 | self.window.set_resizable(False) 51 | 52 | def show_window(self): 53 | self.window.show_all() 54 | Gtk.main() 55 | 56 | def exit_window(self): 57 | self.window.destroy() 58 | Gtk.main_quit() 59 | 60 | def show_successful_login_dialog(self): 61 | dialog = Gtk.MessageDialog(self.window, 0, 62 | Gtk.MessageType.INFO, Gtk.ButtonsType.OK, 63 | "Login Successful") 64 | dialog.format_secondary_text( 65 | "Saving Login Details in configuration file.") 66 | dialog.run() 67 | dialog.destroy() 68 | self.exit_window() 69 | 70 | def show_failed_login_dialog(self): 71 | dialog = Gtk.MessageDialog(self.window, 0, Gtk.MessageType.ERROR, 72 | Gtk.ButtonsType.CANCEL, 73 | "Incorrect Login Details") 74 | dialog.format_secondary_text("Please check your login details again.") 75 | dialog.run() 76 | dialog.destroy() 77 | 78 | def show_connection_error_dialog(self): 79 | dialog = Gtk.MessageDialog(self.window, 0, Gtk.MessageType.ERROR, 80 | Gtk.ButtonsType.CANCEL, "Internet connectivity problem") 81 | dialog.format_secondary_text( 82 | "There is some problem connecting to internet. Please make sure internet is working.") 83 | dialog.run() 84 | dialog.destroy() 85 | 86 | class Handler: 87 | def __init__(self, login_window): 88 | self.login_window = login_window 89 | 90 | def onDeleteWindow(self, *args): 91 | self.login_window.exit_window() 92 | 93 | def signInButtonClicked(self, *args): 94 | COLOR_INVALID = Color(50000, 10000, 10000) 95 | email = self.login_window.email_field.get_text() 96 | password = self.login_window.password_field.get_text() 97 | 98 | result = re.match( 99 | '^[_a-z0-9-]+(\.[_a-z0-9-]+)*@[a-z0-9-]+(\.[a-z0-9-]+)*(\.[a-z]{2,4})$', email) 100 | 101 | if result is None: 102 | self.login_window.email_field.modify_fg( 103 | Gtk.StateFlags.NORMAL, COLOR_INVALID) 104 | return 105 | else: 106 | self.login_window.email_field.modify_fg( 107 | Gtk.StateFlags.NORMAL, None) 108 | 109 | self.login_window.spinner.start() 110 | try: 111 | result = is_valid(email, password) 112 | if result: 113 | self.login_window.spinner.stop() 114 | self.login_window.show_successful_login_dialog() 115 | config['usage_mode'] = 'authenticated' 116 | config['login_credentials']['email'] = email 117 | config['login_credentials']['password'] = password 118 | else: 119 | self.login_window.spinner.stop() 120 | self.login_window.show_failed_login_dialog() 121 | config['usage_mode'] = 'anonymous' 122 | 123 | except ConnectionError: 124 | self.login_window.spinner.stop() 125 | self.login_window.show_connection_error_dialog() 126 | 127 | finally: 128 | self.login_window.spinner.stop() 129 | 130 | def input_changed(self, *args): 131 | email = self.login_window.email_field.get_text() 132 | password = self.login_window.password_field.get_text() 133 | 134 | result = re.match( 135 | '^[_a-z0-9-]+(\.[_a-z0-9-]+)*@[a-z0-9-]+(\.[a-z0-9-]+)*(\.[a-z]{2,4})$', email) 136 | 137 | if result is not None and password is not '': 138 | self.login_window.sign_in_button.set_sensitive(True) 139 | else: 140 | self.login_window.sign_in_button.set_sensitive(False) 141 | -------------------------------------------------------------------------------- /susi_linux/ui/renderer.py: -------------------------------------------------------------------------------- 1 | from abc import ABC, abstractclassmethod 2 | from rx.subject import Subject 3 | from .. import SusiLoop 4 | 5 | 6 | class Renderer(ABC): 7 | def __init__(self): 8 | super().__init__() 9 | self.subject = Subject() 10 | self.susi_loop = SusiLoop(self) 11 | self.susi_loop.start(background = True) 12 | 13 | @abstractclassmethod 14 | def receive_message(self, message_type, payload=None): 15 | pass 16 | 17 | def on_mic_pressed(self): 18 | self.subject.on_next('mic_button_pressed') 19 | -------------------------------------------------------------------------------- /susi_linux/wav/daniel_no_connection_to_the_internet.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/susi_linux/wav/daniel_no_connection_to_the_internet.wav -------------------------------------------------------------------------------- /susi_linux/wav/infobleep.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/susi_linux/wav/infobleep.wav -------------------------------------------------------------------------------- /susi_linux/wav/readme.txt: -------------------------------------------------------------------------------- 1 | These sounds are made on a Mac with (i.e.) 2 | say -v Ting-Ting -o ting-ting_susi_has_started.wave "Susi has started" 3 | 4 | -------------------------------------------------------------------------------- /susi_linux/wav/ting-ting_no_connection_to_the_internet.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/susi_linux/wav/ting-ting_no_connection_to_the_internet.wav -------------------------------------------------------------------------------- /susi_linux/wav/ting-ting_susi_cannot_start.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/susi_linux/wav/ting-ting_susi_cannot_start.wav -------------------------------------------------------------------------------- /susi_linux/wav/ting-ting_susi_has_access_to_the_internet.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/susi_linux/wav/ting-ting_susi_has_access_to_the_internet.wav -------------------------------------------------------------------------------- /susi_linux/wav/ting-ting_susi_has_started.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/susi_linux/wav/ting-ting_susi_has_started.wav -------------------------------------------------------------------------------- /susi_linux/wav/ting-ting_susi_is_alive_and_listening.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/susi_linux/wav/ting-ting_susi_is_alive_and_listening.wav -------------------------------------------------------------------------------- /susi_linux/wav/ting-ting_susi_is_in_startup_mode.wav: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/fossasia/susi_linux/395ea4dbbd2c26cbed8e369b1d8f2af6a3db31a0/susi_linux/wav/ting-ting_susi_is_in_startup_mode.wav -------------------------------------------------------------------------------- /system-integration/desktop/susi-linux-app.desktop.in: -------------------------------------------------------------------------------- 1 | [Desktop Entry] 2 | Name=SUSI.AI Personal Assistant - Application Window 3 | GenericName=SUSI Personal Assistant Application Window 4 | Comment=Privacy aware personal assistant 5 | Exec=@SUSIDIR@/bin/susi-linux-app 6 | Icon=susi-ai 7 | StartupNotify=false 8 | Terminal=false 9 | Type=Application 10 | Categories=Utility; 11 | Keywords=SUSI.AI;Personal Assistant;Office; 12 | -------------------------------------------------------------------------------- /system-integration/desktop/susi-linux-configure.desktop.in: -------------------------------------------------------------------------------- 1 | [Desktop Entry] 2 | Name=SUSI.AI Personal Assistant - Configuration 3 | GenericName=SUSI Personal Assistant Configuration 4 | Comment=Privacy aware personal assistant - configuration 5 | Exec=@SUSIDIR@/bin/susi-linux-configure 6 | Icon=susi-ai 7 | StartupNotify=false 8 | Terminal=false 9 | Type=Application 10 | Categories=Utility; 11 | Keywords=SUSI.AI;Personal Assistant;Office; 12 | -------------------------------------------------------------------------------- /system-integration/scripts/susi-linux: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # Start/Stop the SUSI.AI main program 3 | 4 | # this wrapper is installed as follows: 5 | # user mode 6 | # .../SUSI.AI/bin/ 7 | # .../SUSI.AI/pythonmods/ 8 | # system mode 9 | # prefix/bin/ 10 | # prefix/lib/SUSI.AI/pythonmods 11 | 12 | DIR="$(dirname "$(readlink -f "$0")")" 13 | PMA="$(readlink -m "$DIR/../pythonmods")" 14 | PMB="$(readlink -m "$DIR/../lib/SUSI.AI/pythonmods")" 15 | if [ -d "$PMA" ] && [ -r "$PMA/susi_linux" ] ; then 16 | LOGDIR="$(readlink -m "$DIR/../logs")" 17 | PYTHONPATH="$PMA":$PYTHONPATH 18 | export PYTHONPATH 19 | elif [ -d "$PMB" ] && [ -r "$PMB/susi_linux" ] ; then 20 | LOGDIR=$HOME/.susi.ai/logs 21 | PYTHONPATH="$PMB":$PYTHONPATH 22 | export PYTHONPATH 23 | else 24 | echo "Cannot find SUSI.AI pythonmods, trying without it" >&2 25 | fi 26 | 27 | CMD="python3 -m susi_linux -v --short-log" 28 | 29 | do_start() { 30 | mkdir -p "$LOGDIR" 31 | python3 -m susi_linux -v --short-log > "$LOGDIR/susi-linux.log" 2>&1 & 32 | sleep 3 33 | echo "susi-linux has started, logging to $LOGDIR/susi-linux.log" 34 | } 35 | 36 | do_stop() { 37 | pkill -f susi_linux 38 | } 39 | 40 | case "$1" in 41 | start) 42 | do_start ;; 43 | stop) 44 | do_stop ;; 45 | restart) 46 | do_stop ; sleep 1 ; do_start ;; 47 | *) 48 | echo "Usage: susi-linux {start|stop|restart}" >&2 49 | exit 1 50 | ;; 51 | esac 52 | 53 | 54 | -------------------------------------------------------------------------------- /system-integration/scripts/susi-linux-app: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | # this wrapper is installed as follows: 4 | # user mode 5 | # .../SUSI.AI/bin/ 6 | # .../SUSI.AI/pythonmods/ 7 | # .../SUSI.AI/susi_linux/ 8 | # system mode 9 | # prefix/bin/ 10 | # prefix/lib/SUSI.AI/pythonmods 11 | # prefix/lib/SUSI.AI/susi_linux/ 12 | 13 | DIR="$(dirname "$(readlink -f "$0")")" 14 | SUSIDIR_USER="$(readlink -m "$DIR/..")" 15 | SUSIDIR_SYSTEM="$(readlink -m "$DIR/../lib/SUSI.AI")" 16 | if [ -d "$SUSIDIR_USER" ] && [ -r "$SUSIDIR_USER/pythonmods/susi_linux" ] ; then 17 | PYTHONPATH="$SUSIDIR_USER/pythonmods":$PYTHONPATH 18 | export PYTHONPATH 19 | SUSILINUX="$SUSIDIR_USER/susi_linux" 20 | elif [ -d "$SUSIDIR_SYSTEM" ] && [ -r "$SUSIDIR_SYSTEM/pythonmods/susi_linux" ] ; then 21 | PYTHONPATH="$SUSIDIR_SYSTEM/pythonmods":$PYTHONPATH 22 | export PYTHONPATH 23 | SUSILINUX="$SUSIDIR_SYSTEM/susi_linux" 24 | else 25 | echo "Cannot find SUSI.AI susi_linux folder" >&2 26 | exit 1 27 | fi 28 | exec python3 "$SUSILINUX/app.py" "$@" 29 | 30 | -------------------------------------------------------------------------------- /system-integration/scripts/susi-linux-configure: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # 3 | # this wrapper is installed as follows: 4 | # user mode 5 | # .../SUSI.AI/bin/ 6 | # .../SUSI.AI/pythonmods/ 7 | # .../SUSI.AI/susi_linux/ 8 | # system mode 9 | # prefix/bin/ 10 | # prefix/lib/SUSI.AI/pythonmods 11 | # prefix/lib/SUSI.AI/susi_linux/ 12 | 13 | DIR="$(dirname "$(readlink -f "$0")")" 14 | SUSIDIR_USER="$(readlink -m "$DIR/..")" 15 | SUSIDIR_SYSTEM="$(readlink -m "$DIR/../lib/SUSI.AI")" 16 | if [ -d "$SUSIDIR_USER" ] && [ -r "$SUSIDIR_USER/pythonmods/susi_linux" ] ; then 17 | PYTHONPATH="$SUSIDIR_USER/pythonmods":$PYTHONPATH 18 | export PYTHONPATH 19 | SUSILINUX="$SUSIDIR_USER/susi_linux" 20 | elif [ -d "$SUSIDIR_SYSTEM" ] && [ -r "$SUSIDIR_SYSTEM/pythonmods/susi_linux" ] ; then 21 | PYTHONPATH="$SUSIDIR_SYSTEM/pythonmods":$PYTHONPATH 22 | export PYTHONPATH 23 | SUSILINUX="$SUSIDIR_SYSTEM/susi_linux" 24 | else 25 | echo "Cannot find SUSI.AI susi_linux folder" >&2 26 | exit 1 27 | fi 28 | exec python3 "$SUSILINUX/configure.py" "$@" 29 | 30 | -------------------------------------------------------------------------------- /system-integration/systemd/ss-susi-linux.service.in: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=SUSI Linux 3 | Wants=network-online.target 4 | After=network-online.target ss-susi-server.service 5 | 6 | [Service] 7 | Type=idle 8 | ExecStart=@SUSIDIR@/bin/susi-linux start 9 | Restart=on-failure 10 | 11 | [Install] 12 | WantedBy=multi-user.target 13 | 14 | -------------------------------------------------------------------------------- /system-integration/systemd/ss-susi-linux@.service.in: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=SUSI Linux 3 | Wants=network-online.target 4 | After=network-online.target ss-susi-server.service 5 | 6 | [Service] 7 | Type=idle 8 | User=%i 9 | ExecStart=@SUSIDIR@/bin/susi-linux start 10 | Restart=on-failure 11 | 12 | [Install] 13 | WantedBy=multi-user.target 14 | 15 | --------------------------------------------------------------------------------