├── requirements.txt
├── .github
    └── ISSUE_TEMPLATE
    │   ├── config.yml
    │   ├── enhancement.yml
    │   └── bug.yml
├── start.sh
├── SECURITY.md
├── README.md
├── slack_helpers.py
├── setup.md
├── vectara_functions.py
├── LICENSE
└── slackbot.py


/requirements.txt:
--------------------------------------------------------------------------------
1 | slack_bolt==1.14.3
2 | requests==2.25.1
3 | Authlib==1.0.1


--------------------------------------------------------------------------------
/.github/ISSUE_TEMPLATE/config.yml:
--------------------------------------------------------------------------------
1 | blank_issues_enabled: true
2 | contact_links:
3 |   - name: Questions
4 |     url: https://discuss.vectara.com/
5 |     about: Ask questions and interact with the community here!
6 | 


--------------------------------------------------------------------------------
/start.sh:
--------------------------------------------------------------------------------
1 | export SLACK_BOT_TOKEN=
2 | export SLACK_APP_TOKEN=
3 | export SLACK_WORKSPACE_SUBDOMAIN=
4 | export VECTARA_CUSTOMER_ID=
5 | export VECTARA_CORPUS_ID=
6 | export VECTARA_APP_ID=
7 | export VECTARA_APP_SECRET=
8 | export VECTARA_USE_RERANKER=true
9 | python3 slackbot.py


--------------------------------------------------------------------------------
/SECURITY.md:
--------------------------------------------------------------------------------
 1 | # Security Policy
 2 | 
 3 | The Vectara trust and security center, including our security policy, can be found at
 4 | [https://vectara.com/legal/security-at-vectara/](https://vectara.com/legal/security-at-vectara/).
 5 | 
 6 | 
 7 | ## Reporting a Vulnerability
 8 | 
 9 | Please send security vulnerability reports to support@vectara.com.
10 | 


--------------------------------------------------------------------------------
/.github/ISSUE_TEMPLATE/enhancement.yml:
--------------------------------------------------------------------------------
 1 | name: 💡 Enhancement
 2 | description: Request a new feature
 3 | labels: ["enhancement"]
 4 | body:
 5 | - type: textarea
 6 |   attributes:
 7 |     label: Requested Feature
 8 |     description: Describe what feature/enhancement you would like to see
 9 |     placeholder: |
10 |       When I do <X>, I would like <Y> to happen because <Z>
11 |       ```...```
12 |   validations:
13 |     required: true
14 | - type: textarea
15 |   attributes:
16 |     label: Anything else?
17 |     description: |
18 |       - Use case
19 |       - Python version
20 |       - Would you be willing to try implementing the feature?
21 |   validations:
22 |     required: false
23 | 


--------------------------------------------------------------------------------
/.github/ISSUE_TEMPLATE/bug.yml:
--------------------------------------------------------------------------------
 1 | name: 🐞 Bug
 2 | description: Something is not working as indended.
 3 | labels: ["bug"]
 4 | body:
 5 | - type: textarea
 6 |   attributes:
 7 |     label: Current Behavior
 8 |     description: What behavior you're experiencing.
 9 |     placeholder: |
10 |       When I do <X>, <Y> happens and I see the error message attached below:
11 |       ```...```
12 |   validations:
13 |     required: false
14 | - type: textarea
15 |   attributes:
16 |     label: Expected Behavior
17 |     description: What you expected to happen.
18 |     placeholder: When I do <X>, <Z> should happen instead.
19 |   validations:
20 |     required: false
21 | - type: textarea
22 |   attributes:
23 |     label: Steps To Reproduce
24 |     description: Steps to reproduce the behavior.
25 |     placeholder: |
26 |       1. In this environment...
27 |       2. With this config...
28 |       3. Run '...'
29 |       4. See error...
30 |     render: markdown
31 |   validations:
32 |     required: false
33 | - type: textarea
34 |   attributes:
35 |     label: Anything else?
36 |     description: |
37 |       - Operating system
38 |       - Python version
39 |   validations:
40 |     required: false
41 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
 1 | # What is it
 2 | This is a bot for [Slack](https://slack.com/) which indexes messages into
 3 | [Vectara's](http://vectara.com/) platform and makes them available for Search.
 4 | 
 5 | The difference between this and Slack's built-in search capabilities is that
 6 | this adds interactive semantic/neural search capabilities.  You ask the bot a
 7 | question and it returns a link to what it knows, as well as interactive filters
 8 | for dates, channels, and/or people.  It performs the search in public, allowing
 9 | all users to see the relevant result(s) at the same time.
10 | 
11 | This particular bot is primarily intended as a demonstration of the Vectara
12 | platform, but can be used practically to add multi-lingual neural search to
13 | your Slack workspace.
14 | 
15 | For an example of what this bot can do, see the following, which has no
16 | substantial keyword overlap:
17 | 
18 | ![Example Search](img/example.png)
19 | 
20 | # How to use it
21 | Have a look at [setup.md](setup.md) for instructions on how to set up the bot.
22 | After you have it set up, invite it to one or more channels.  After it's been
23 | added, continue using the Slack channel(s) as normal.  If you want to perform a
24 | search, send a message in the channel to `@your_bot_name` and it will respond
25 | with the top search result.
26 | 
27 | # Limitations
28 | This Slackbot is currently *mostly* intended to be an example of how to
29 | use/interact with the Vectara platform via the APIs and with a more
30 | conversational feel than many search engines.  However, there are some
31 | limitations which are worth noting if you're going to use this, many of which
32 | we'd like to address over time.
33 | 
34 | Current limitations include:
35 | - The bot doesn't index any content that was created *prior* to the
36 | bot being invited to the channel.  So it's not yet possible to search
37 | "historic" content.
38 | - The bot doesn't index images or files.  Vectara's platform does
39 | support indexing files, but the Slack-specific interactions and file indexing
40 | have not been added.
41 | - The bot doesn't keep/index threads/contexts.  So if you're responding to a
42 | message, it doesn't currently know that something was a response or what
43 | messages were added to a thread
44 | - Slack does not send/resolve any usernames or channel names directly.  This
45 | helps preserve some anonymity, but means you can't ask it "What did Shane say
46 | about foo?"
47 | - Messages for "Growth" plan users are not instantly indexed, but instead
48 | may undergo a delay from message time to indexing.  So searches for very recent
49 | messages may fail/produce irrelevant results until the recent message is
50 | indexed.
51 | 
52 | Pull requests are welcome for the above or any other issues/enhancements!
53 | 
54 | ## License
55 | This code is licensed Apache 2.0.  For more details, see the [license file](LICENSE)


--------------------------------------------------------------------------------
/slack_helpers.py:
--------------------------------------------------------------------------------
 1 | import re
 2 | 
 3 | def homepage_blocks():
 4 |     return ([
 5 |           {
 6 |             "type": "section",
 7 |             "text": {
 8 |               "type": "mrkdwn",
 9 |               "text": "*Welcome to Vectara* :tada:"
10 |             }
11 |           },
12 |           {
13 |             "type": "divider"
14 |           },
15 |           {
16 |             "type": "section",
17 |             "text": {
18 |               "type": "mrkdwn",
19 |               "text": "To interact with Vectara with this slackbot:\n1. Invite it to any channels you want to be searchable\n2. Wait :slightly_smiling_face:"
20 |             }
21 |           },
22 |           {
23 |             "type": "section",
24 |             "text": {
25 |               "type": "mrkdwn",
26 |               "text": "The slackbot doesn't attempt to index any message history prior to it joining: it will only index Slack messages sent after the bot is in the channel"
27 |             }
28 |           },
29 |           {
30 |             "type": "divider"
31 |           },
32 |           {
33 |             "type": "section",
34 |             "text": {
35 |               "type": "mrkdwn",
36 |               "text": "After messages have been sent while the bot has been in the channels, you can perform searches several ways:\n1. Send me a message: `@` me and then send query text e.g. `@VectaraSlackSearch who mentioned going to Hawaii?`\n2. Use the `/vectara` command, e.g. `/vectara who mentioned going to Hawaii?`\n3. Use the `/vectara` command with channel parameters, e.g. `/vectara #vacations who mentioned going to Hawaii` to limit the search to just that channel"
37 |             }
38 |           }
39 |         ])
40 | 
41 | def escape_markdown(text, *, as_needed=False, ignore_links=True):
42 |     """A helper function that escapes Discord's/Slack's markdown.
43 | 
44 |     Parameters
45 |     -----------
46 |     text: :class:`str`
47 |         The text to escape markdown from.
48 |     as_needed: :class:`bool`
49 |         Whether to escape the markdown characters as needed. This
50 |         means that it does not escape extraneous characters if it's
51 |         not necessary, e.g. ``**hello**`` is escaped into ``\*\*hello**``
52 |         instead of ``\*\*hello\*\*``. Note however that this can open
53 |         you up to some clever syntax abuse. Defaults to ``False``.
54 |     ignore_links: :class:`bool`
55 |         Whether to leave links alone when escaping markdown. For example,
56 |         if a URL in the text contains characters such as ``_`` then it will
57 |         be left alone. This option is not supported with ``as_needed``.
58 |         Defaults to ``True``.
59 | 
60 |     Returns
61 |     --------
62 |     :class:`str`
63 |         The text with the markdown special characters escaped with a slash.
64 |     """
65 | 
66 |     # constants
67 |     _MARKDOWN_ESCAPE_SUBREGEX = '|'.join(r'\{0}(?=([\s\S]*((?<!\{0})\{0})))'.format(c)
68 |                                                 for c in ('*', '`', '_', '~', '|'))
69 |     _MARKDOWN_ESCAPE_COMMON = r'^>(?:>>)?\s|\[.+\]\(.+\)'
70 |     _MARKDOWN_ESCAPE_REGEX = re.compile(r'(?P<markdown>%s|%s)' % (_MARKDOWN_ESCAPE_SUBREGEX, _MARKDOWN_ESCAPE_COMMON))
71 | 
72 |     if not as_needed:
73 |         url_regex = r'(?P<url><[^: >]+:\/[^ >]+>|(?:https?|steam):\/\/[^\s<]+[^<.,:;\"\'\]\s])'
74 |         def replacement(match):
75 |             groupdict = match.groupdict()
76 |             is_url = groupdict.get('url')
77 |             if is_url:
78 |                 return is_url
79 |             return '\\' + groupdict['markdown']
80 | 
81 |         regex = r'(?P<markdown>[_\\~|\*`]|%s)' % _MARKDOWN_ESCAPE_COMMON
82 |         if ignore_links:
83 |             regex = '(?:%s|%s)' % (url_regex, regex)
84 |         return re.sub(regex, replacement, text)
85 |     else:
86 |         text = re.sub(r'\\', r'\\\\', text)
87 |         return _MARKDOWN_ESCAPE_REGEX.sub(r'\\\1', text) 


--------------------------------------------------------------------------------
/setup.md:
--------------------------------------------------------------------------------
  1 | In oroder to set up an instance of this Slack bot, you'll need to do a few
  2 | things first.
  3 | 
  4 | # Clone This Repo and Add Dependencies
  5 | Once you've cloned this repo, the Python package requirements can be found in
  6 | [requirements.txt](requirements.txt).  Install these packages before
  7 | continuing.
  8 | 
  9 | # Get a Vectara Account
 10 | If you don't already have a [Vectara](https://vectara.com) account, you'll need
 11 | one, so go there and sign up first.  The bot uses Vectara for search and
 12 | filtering capabilities.
 13 | 
 14 | ## Customer ID
 15 | Your Vectara customer ID to will be in your welcome e-mail from Vectara.
 16 | Alternatively, you can copy it by clicking your username in the upper-right
 17 | hand corner of the product console.  Once you've got this number, paste it into
 18 | `start.sh` in the value for VECTARA_CUSTOMER_ID
 19 | 
 20 | ## Corpus Creation
 21 | From [Vectara](https://console.vectara.com), create a new corpus.  This corpus
 22 | will hold the content for our Slack messages.
 23 | 
 24 | - Name the corpus anything of your choosing
 25 | - In the section "Filter Attributes:
 26 |   -  Add a new filter attibute with `Name` of `poster`, `level` of `Document`
 27 |   and `Data Type` of `Text`
 28 |   -  Add a new filter attibute with `Name` of `channel`, `level` of `Document`
 29 |   and `Data Type` of `Text`
 30 |   -  Add a new filter attibute with `Name` of `timestamp`, `level` of
 31 |   `Document` and `Data Type` of `Float`
 32 | - Optionally, give the corpus a description
 33 | 
 34 | Once you've created the corpus, navigate to it and copy the `ID` value to
 35 | `VECTARA_CORPUS_ID` in `start.sh`
 36 | 
 37 | ## Vectara Authentication
 38 | Go to [Vectara's Authentication](https://console.vectara.com/console/authentication)
 39 | 
 40 | Create a new App Client and give it any name/description of your choosing.  You
 41 | can leave all other fields at their default values.
 42 | 
 43 | Once you've created the app client, copy the string in the table under the name
 44 | of the app client and paste it to `VECTARA_APP_ID` in `start.sh`
 45 | 
 46 | Next, click the `...` on the right, go to "Show Secret."  Copy the secret value
 47 | add paste to `VECTARA_APP_SECRET` in `start.sh`
 48 | 
 49 | ## Corpus Authorization
 50 | Go back to the corpus you created in the `Corpus Creation` step and go to the
 51 | Authorization tab. Click "Create Role" and select the App Client you created
 52 | in the previous step. Give it `ADM` permissions to be able to both index and
 53 | search the corpus.
 54 | 
 55 | # Create a New Slack App
 56 | Go to https://api.slack.com/apps?new_app=1 and create an app "From scratch."
 57 | Name the app whatever you want, and select a workspace that you intend to
 58 | install it to.
 59 | 
 60 | Once you've done this, click "Create App."
 61 | 
 62 | # Slack App Settings
 63 | 
 64 | ## Socket Mode
 65 | Under `Settings` in the navigation bar, select `Socket Mode` and toggle *ON*
 66 | socket mode for this application.
 67 | 
 68 | When you enable Socket Mode, Slack will give you an application token that our
 69 | bot will need.  This token will typically start with `xapp`.  Copy this token
 70 | and set it in `start.sh` as the value for `SLACK_APP_TOKEN`.
 71 | 
 72 | ## Event Subscriptions
 73 | Under `Features` in the navigation bar, select `Event Subscriptions` and ensure
 74 | `Enable Events` is turned *ON*.  The navigate to the section under "Subscribe
 75 | to bot events" and add subscriptions for:
 76 | - message.groups
 77 | - message.im
 78 | - message.channels
 79 | 
 80 | Once you've done this, click "Save Changes"
 81 | 
 82 | ## OAuth & Permissions
 83 | Under `Features` in the navigation bar, go to `OAuth & Permissions`.
 84 | 
 85 | You'll need to enable the `chat:write` permission under the `Scopes` section.
 86 | 
 87 | After you've done this, install the app in Slack by going to `Install App`
 88 | under `Settings` in the navigation bar.  Copy the OAuth token value that's
 89 | provided to you when you install.  It generally starts with `xoab`.  Paste the
 90 | value o fthis into the `SLACK_BOT_TOKEN` value in `start.sh`
 91 | 
 92 | ## Workspace
 93 | Set your Slack workspace as `SLACK_WORKSPACE_SUBDOMAIN` in `start.sh`.  For
 94 | example, if you log into Slack as `foo.slack.com`, then your
 95 | `SLACK_WORKSPACE_SUBDOMAIN` would be `foo`
 96 | 
 97 | ## Invite
 98 | The last step in Slack is to invite the bot to channels you want to be
 99 | searchable.
100 | 


--------------------------------------------------------------------------------
/vectara_functions.py:
--------------------------------------------------------------------------------
  1 | import json
  2 | import logging
  3 | import os
  4 | import requests
  5 | from authlib.integrations.requests_client import OAuth2Session
  6 | 
  7 | def _get_jwt_token(auth_url: str = None):
  8 |     """Connect to the server and get a JWT token."""
  9 | 
 10 |     if os.environ.get('VECTARA_AUTH_URL') != None:
 11 |         auth_url = os.environ.get('VECTARA_AUTH_URL')
 12 |     if auth_url == None:
 13 |       auth_url = "https://vectara-prod-{}.auth.us-west-2.amazoncognito.com".format(os.environ.get('VECTARA_CUSTOMER_ID'))
 14 | 
 15 |     token_endpoint = f"{auth_url}/oauth2/token"
 16 |     session = OAuth2Session(
 17 |         os.environ.get('VECTARA_APP_ID'), os.environ.get('VECTARA_APP_SECRET'), scope="")
 18 |     token = session.fetch_token(token_endpoint, grant_type="client_credentials")
 19 |     return token["access_token"]
 20 | 
 21 | def search_raw(headers: dict, data: dict):
 22 |     """ Takes headers and the JSON body and performs a search against Vectara """
 23 |     payload = json.dumps(data)
 24 |     logging.debug("Raw search payload: %s",payload)
 25 |     
 26 |     response = requests.post(
 27 |         "https://api.vectara.io/v1/query",
 28 |         data=payload,
 29 |         verify=True,
 30 |         headers=headers)
 31 |     search_results = json.loads(response.content)
 32 |     return data, search_results
 33 | 
 34 | def search(search_text: str, rerank: bool, num_results: int, metadata_filters: list = None):
 35 |     """ Takes headers and the JSON body and performs a search against Vectara """
 36 |     jwt_token = _get_jwt_token()
 37 |     api_key_header = {
 38 |         "Authorization": f"Bearer {jwt_token}",
 39 |         "customer-id": os.environ.get('VECTARA_CUSTOMER_ID')
 40 |     }
 41 |     data_dict = {
 42 |         "query": [
 43 |             {
 44 |                 "query": search_text,
 45 |                 "num_results": num_results,
 46 |                 "corpus_key": [
 47 |                     {
 48 |                         "customer_id": int(os.environ.get('VECTARA_CUSTOMER_ID')),
 49 |                         "corpus_id": int(os.environ.get('VECTARA_CORPUS_ID')),
 50 |                         "metadata_filter": "part.is_title IS NULL"
 51 |                     }
 52 |                 ]
 53 |             }
 54 |         ]
 55 |     }
 56 |     if rerank == True:
 57 |         data_dict['query'][0]['rerankingConfig'] = { "reranker_id": 272725717 }
 58 |         data_dict['query'][0]['start'] = 0
 59 |         data_dict['query'][0]['num_results'] = 100
 60 |     if metadata_filters != None:
 61 |         metadata_filters.extend(['part.is_title IS NULL'])
 62 |         filter_string = " AND ".join(metadata_filters)
 63 |         data_dict['query'][0]['corpus_key'][0]['metadata_filter'] = filter_string
 64 |     return search_raw(api_key_header, data_dict)
 65 | 
 66 | def index_message(customer_id: int, corpus_id: int, text: str,
 67 |                   id: str, title: str, metadata: dict = None,
 68 |                   idx_address: str = "api.vectara.io"):
 69 |     """ Indexes a document to Vectara """
 70 |     jwt_token = _get_jwt_token()
 71 |     post_headers = {
 72 |         "Authorization": f"Bearer {jwt_token}",
 73 |         "customer-id": f"{customer_id}"
 74 |     }
 75 | 
 76 |     document = {}
 77 |     document["document_id"] = id
 78 |     # Note that the document ID must be unique for a given corpus
 79 |     document["title"] = title
 80 |     document["metadata_json"] = json.dumps(metadata)
 81 |     sections = []
 82 |     section = {}
 83 |     section["text"] = text
 84 |     sections.append(section)
 85 |     document["section"] = sections
 86 | 
 87 |     request = {}
 88 |     request['customer_id'] = customer_id
 89 |     request['corpus_id'] = corpus_id
 90 |     request['document'] = document
 91 | 
 92 |     logging.debug("Raw indexing request: %s",json.dumps(request))
 93 | 
 94 |     response = requests.post(
 95 |         f"https://{idx_address}/v1/index",
 96 |         data=json.dumps(request),
 97 |         verify=True,
 98 |         headers=post_headers)
 99 |     logging.debug("Raw indexing response: %s", response)
100 | 
101 |     if response.status_code != 200:
102 |         logging.error("REST upload failed with code %d, reason %s, text %s",
103 |                        response.status_code,
104 |                        response.reason,
105 |                        response.text)
106 |         return response, False
107 |     return response, True
108 | 
109 | def get_metadata_value(document_metadata, metadata_name):
110 |   """ This function takes the name of metadata and returns its value or 'Unknown' """
111 |   val = None
112 |   try:
113 |     val = list(filter(lambda x: x['name'] == metadata_name, document_metadata))[0]['value']
114 |   except:
115 |     val = "Unknown"
116 |   return val


--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
  1 |                                  Apache License
  2 |                            Version 2.0, January 2004
  3 |                         http://www.apache.org/licenses/
  4 | 
  5 |    TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
  6 | 
  7 |    1. Definitions.
  8 | 
  9 |       "License" shall mean the terms and conditions for use, reproduction,
 10 |       and distribution as defined by Sections 1 through 9 of this document.
 11 | 
 12 |       "Licensor" shall mean the copyright owner or entity authorized by
 13 |       the copyright owner that is granting the License.
 14 | 
 15 |       "Legal Entity" shall mean the union of the acting entity and all
 16 |       other entities that control, are controlled by, or are under common
 17 |       control with that entity. For the purposes of this definition,
 18 |       "control" means (i) the power, direct or indirect, to cause the
 19 |       direction or management of such entity, whether by contract or
 20 |       otherwise, or (ii) ownership of fifty percent (50%) or more of the
 21 |       outstanding shares, or (iii) beneficial ownership of such entity.
 22 | 
 23 |       "You" (or "Your") shall mean an individual or Legal Entity
 24 |       exercising permissions granted by this License.
 25 | 
 26 |       "Source" form shall mean the preferred form for making modifications,
 27 |       including but not limited to software source code, documentation
 28 |       source, and configuration files.
 29 | 
 30 |       "Object" form shall mean any form resulting from mechanical
 31 |       transformation or translation of a Source form, including but
 32 |       not limited to compiled object code, generated documentation,
 33 |       and conversions to other media types.
 34 | 
 35 |       "Work" shall mean the work of authorship, whether in Source or
 36 |       Object form, made available under the License, as indicated by a
 37 |       copyright notice that is included in or attached to the work
 38 |       (an example is provided in the Appendix below).
 39 | 
 40 |       "Derivative Works" shall mean any work, whether in Source or Object
 41 |       form, that is based on (or derived from) the Work and for which the
 42 |       editorial revisions, annotations, elaborations, or other modifications
 43 |       represent, as a whole, an original work of authorship. For the purposes
 44 |       of this License, Derivative Works shall not include works that remain
 45 |       separable from, or merely link (or bind by name) to the interfaces of,
 46 |       the Work and Derivative Works thereof.
 47 | 
 48 |       "Contribution" shall mean any work of authorship, including
 49 |       the original version of the Work and any modifications or additions
 50 |       to that Work or Derivative Works thereof, that is intentionally
 51 |       submitted to Licensor for inclusion in the Work by the copyright owner
 52 |       or by an individual or Legal Entity authorized to submit on behalf of
 53 |       the copyright owner. For the purposes of this definition, "submitted"
 54 |       means any form of electronic, verbal, or written communication sent
 55 |       to the Licensor or its representatives, including but not limited to
 56 |       communication on electronic mailing lists, source code control systems,
 57 |       and issue tracking systems that are managed by, or on behalf of, the
 58 |       Licensor for the purpose of discussing and improving the Work, but
 59 |       excluding communication that is conspicuously marked or otherwise
 60 |       designated in writing by the copyright owner as "Not a Contribution."
 61 | 
 62 |       "Contributor" shall mean Licensor and any individual or Legal Entity
 63 |       on behalf of whom a Contribution has been received by Licensor and
 64 |       subsequently incorporated within the Work.
 65 | 
 66 |    2. Grant of Copyright License. Subject to the terms and conditions of
 67 |       this License, each Contributor hereby grants to You a perpetual,
 68 |       worldwide, non-exclusive, no-charge, royalty-free, irrevocable
 69 |       copyright license to reproduce, prepare Derivative Works of,
 70 |       publicly display, publicly perform, sublicense, and distribute the
 71 |       Work and such Derivative Works in Source or Object form.
 72 | 
 73 |    3. Grant of Patent License. Subject to the terms and conditions of
 74 |       this License, each Contributor hereby grants to You a perpetual,
 75 |       worldwide, non-exclusive, no-charge, royalty-free, irrevocable
 76 |       (except as stated in this section) patent license to make, have made,
 77 |       use, offer to sell, sell, import, and otherwise transfer the Work,
 78 |       where such license applies only to those patent claims licensable
 79 |       by such Contributor that are necessarily infringed by their
 80 |       Contribution(s) alone or by combination of their Contribution(s)
 81 |       with the Work to which such Contribution(s) was submitted. If You
 82 |       institute patent litigation against any entity (including a
 83 |       cross-claim or counterclaim in a lawsuit) alleging that the Work
 84 |       or a Contribution incorporated within the Work constitutes direct
 85 |       or contributory patent infringement, then any patent licenses
 86 |       granted to You under this License for that Work shall terminate
 87 |       as of the date such litigation is filed.
 88 | 
 89 |    4. Redistribution. You may reproduce and distribute copies of the
 90 |       Work or Derivative Works thereof in any medium, with or without
 91 |       modifications, and in Source or Object form, provided that You
 92 |       meet the following conditions:
 93 | 
 94 |       (a) You must give any other recipients of the Work or
 95 |           Derivative Works a copy of this License; and
 96 | 
 97 |       (b) You must cause any modified files to carry prominent notices
 98 |           stating that You changed the files; and
 99 | 
100 |       (c) You must retain, in the Source form of any Derivative Works
101 |           that You distribute, all copyright, patent, trademark, and
102 |           attribution notices from the Source form of the Work,
103 |           excluding those notices that do not pertain to any part of
104 |           the Derivative Works; and
105 | 
106 |       (d) If the Work includes a "NOTICE" text file as part of its
107 |           distribution, then any Derivative Works that You distribute must
108 |           include a readable copy of the attribution notices contained
109 |           within such NOTICE file, excluding those notices that do not
110 |           pertain to any part of the Derivative Works, in at least one
111 |           of the following places: within a NOTICE text file distributed
112 |           as part of the Derivative Works; within the Source form or
113 |           documentation, if provided along with the Derivative Works; or,
114 |           within a display generated by the Derivative Works, if and
115 |           wherever such third-party notices normally appear. The contents
116 |           of the NOTICE file are for informational purposes only and
117 |           do not modify the License. You may add Your own attribution
118 |           notices within Derivative Works that You distribute, alongside
119 |           or as an addendum to the NOTICE text from the Work, provided
120 |           that such additional attribution notices cannot be construed
121 |           as modifying the License.
122 | 
123 |       You may add Your own copyright statement to Your modifications and
124 |       may provide additional or different license terms and conditions
125 |       for use, reproduction, or distribution of Your modifications, or
126 |       for any such Derivative Works as a whole, provided Your use,
127 |       reproduction, and distribution of the Work otherwise complies with
128 |       the conditions stated in this License.
129 | 
130 |    5. Submission of Contributions. Unless You explicitly state otherwise,
131 |       any Contribution intentionally submitted for inclusion in the Work
132 |       by You to the Licensor shall be under the terms and conditions of
133 |       this License, without any additional terms or conditions.
134 |       Notwithstanding the above, nothing herein shall supersede or modify
135 |       the terms of any separate license agreement you may have executed
136 |       with Licensor regarding such Contributions.
137 | 
138 |    6. Trademarks. This License does not grant permission to use the trade
139 |       names, trademarks, service marks, or product names of the Licensor,
140 |       except as required for reasonable and customary use in describing the
141 |       origin of the Work and reproducing the content of the NOTICE file.
142 | 
143 |    7. Disclaimer of Warranty. Unless required by applicable law or
144 |       agreed to in writing, Licensor provides the Work (and each
145 |       Contributor provides its Contributions) on an "AS IS" BASIS,
146 |       WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147 |       implied, including, without limitation, any warranties or conditions
148 |       of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149 |       PARTICULAR PURPOSE. You are solely responsible for determining the
150 |       appropriateness of using or redistributing the Work and assume any
151 |       risks associated with Your exercise of permissions under this License.
152 | 
153 |    8. Limitation of Liability. In no event and under no legal theory,
154 |       whether in tort (including negligence), contract, or otherwise,
155 |       unless required by applicable law (such as deliberate and grossly
156 |       negligent acts) or agreed to in writing, shall any Contributor be
157 |       liable to You for damages, including any direct, indirect, special,
158 |       incidental, or consequential damages of any character arising as a
159 |       result of this License or out of the use or inability to use the
160 |       Work (including but not limited to damages for loss of goodwill,
161 |       work stoppage, computer failure or malfunction, or any and all
162 |       other commercial damages or losses), even if such Contributor
163 |       has been advised of the possibility of such damages.
164 | 
165 |    9. Accepting Warranty or Additional Liability. While redistributing
166 |       the Work or Derivative Works thereof, You may choose to offer,
167 |       and charge a fee for, acceptance of support, warranty, indemnity,
168 |       or other liability obligations and/or rights consistent with this
169 |       License. However, in accepting such obligations, You may act only
170 |       on Your own behalf and on Your sole responsibility, not on behalf
171 |       of any other Contributor, and only if You agree to indemnify,
172 |       defend, and hold each Contributor harmless for any liability
173 |       incurred by, or claims asserted against, such Contributor by reason
174 |       of your accepting any such warranty or additional liability.
175 | 
176 |    END OF TERMS AND CONDITIONS
177 | 
178 |    APPENDIX: How to apply the Apache License to your work.
179 | 
180 |       To apply the Apache License to your work, attach the following
181 |       boilerplate notice, with the fields enclosed by brackets "[]"
182 |       replaced with your own identifying information. (Don't include
183 |       the brackets!)  The text should be enclosed in the appropriate
184 |       comment syntax for the file format. We also recommend that a
185 |       file or class name and description of purpose be included on the
186 |       same "printed page" as the copyright notice for easier
187 |       identification within third-party archives.
188 | 
189 |    Copyright [yyyy] [name of copyright owner]
190 | 
191 |    Licensed under the Apache License, Version 2.0 (the "License");
192 |    you may not use this file except in compliance with the License.
193 |    You may obtain a copy of the License at
194 | 
195 |        http://www.apache.org/licenses/LICENSE-2.0
196 | 
197 |    Unless required by applicable law or agreed to in writing, software
198 |    distributed under the License is distributed on an "AS IS" BASIS,
199 |    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200 |    See the License for the specific language governing permissions and
201 |    limitations under the License.
202 | 


--------------------------------------------------------------------------------
/slackbot.py:
--------------------------------------------------------------------------------
  1 | import json
  2 | import logging
  3 | import os
  4 | from datetime import datetime, timedelta
  5 | from slack_bolt import App
  6 | from slack_bolt.adapter.socket_mode import SocketModeHandler
  7 | from vectara_functions import get_metadata_value, index_message, search, search_raw
  8 | from slack_helpers import homepage_blocks, escape_markdown
  9 | 
 10 | app = App(token=os.environ.get('SLACK_BOT_TOKEN'))
 11 | 
 12 | def get_channel_filter(channel):
 13 |   return 'doc.channel = \'{}\''.format(channel)
 14 | 
 15 | def get_user_filter(user):
 16 |   return 'doc.poster = \'{}\''.format(user)
 17 | 
 18 | def get_start_date_filter(date):
 19 |   utc_time = datetime.strptime(date, "%Y-%m-%d")
 20 |   epoch_time = (utc_time - datetime(1970, 1, 1)).total_seconds()
 21 |   return 'doc.timestamp >= {}'.format(epoch_time)
 22 | 
 23 | def get_end_date_filter(date):
 24 |   utc_time = datetime.strptime(date, "%Y-%m-%d")
 25 |   epoch_time = (utc_time - datetime(1970, 1, 1)).total_seconds()
 26 |   return 'doc.timestamp <= {}'.format(epoch_time)
 27 | 
 28 | def extract_filters_from_state(state):
 29 |   filter_values = []
 30 |   if state != None:
 31 |       for block in state['values']:
 32 |         filters = state['values'][block]
 33 |         for filter in filters:
 34 |           if filter == 'filter_by_channel' and state['values'][block][filter]['selected_channel'] != None:
 35 |             filter_values['channel'] = get_channel_filter(state['values'][block][filter]['selected_channel'])
 36 |           elif filter == 'filter_by_user' and state['values'][block][filter]['selected_user'] != None:
 37 |             filter_values['user'] = get_user_filter(state['values'][block][filter]['selected_user'])
 38 |           elif filter == 'filter_start_date':
 39 |             filter_values['start_date'] = get_start_date_filter(state['values'][block][filter]['selected_date'])
 40 |           elif filter == 'filter_end_date':
 41 |             filter_values['end_date'] = get_end_date_filter(state['values'][block][filter]['selected_date'])
 42 |   return filter_values
 43 | 
 44 | def get_original_query_text(message):
 45 |   """Extracts the original query text by looking up"""
 46 |   # TODO: create a better way to grab the original query text
 47 |   for b in message['blocks']:
 48 |     if 'text' in b and 'text' in b['text'] and b['text']['text'].startswith('Search results for:'):
 49 |       original_text = b['text']['text'].replace('Search results for: ','')
 50 |       original_text = original_text.strip('*')
 51 |       return original_text
 52 |   return None
 53 | 
 54 | def standard_query_and_filter(ack, body, say, logger, extract_filters_from_state = True):
 55 |   """Helper function for most of the filters.
 56 |   - Extracts the filter state
 57 |   - Pulls the original query text
 58 |   - Performs the search
 59 |   - Responds to the user
 60 |   """
 61 |   ack()
 62 |   user = None
 63 |   channel = None
 64 |   start_date = None
 65 |   end_date = None
 66 |   if extract_filters_from_state:
 67 |     filters = extract_filters_from_state(body['state'])
 68 |     user = filters['user'] if 'user' in filters else None
 69 |     channel = filters['channel'] if 'channel' in filters else None
 70 |     start_date = filters['start_date'] if 'start_date' in filters else None
 71 |     end_date = filters['end_date'] if 'end_date' in filters else None
 72 | 
 73 |   original_search = get_original_query_text(body['message'])
 74 |   query_and_respond(say, original_search,
 75 |                     start_date=start_date, end_date=end_date,
 76 |                     filter_by_user=user, filter_by_channel=channel)
 77 | 
 78 | @app.action("filter_by_channel")
 79 | def filter_by_channel(ack, body, say, logger):
 80 |   """Triggered when a user asks for the results to be filtered to a specific channel."""
 81 |   standard_query_and_filter(ack, body, say, logger)
 82 | 
 83 | @app.action("filter_by_user")
 84 | def filter_by_user(ack, body, say, logger):
 85 |   """Triggered when a user asks for the results to be filtered to a specific user."""
 86 |   standard_query_and_filter(ack, body, say, logger)
 87 | 
 88 | @app.action("filter_start_date")
 89 | def filter_by_start_date(ack, body, say, logger):
 90 |   """Triggered when a user asks for the results to be filtered >= some date."""
 91 |   standard_query_and_filter(ack, body, say, logger)
 92 | 
 93 | @app.action("filter_end_date")
 94 | def filter_by_end_date(ack, body, say, logger):
 95 |   """Triggered when a user asks for the results to be filtered <= some date."""
 96 |   standard_query_and_filter(ack, body, say, logger)
 97 | 
 98 | @app.action("more_results")
 99 | def more_results(ack, body, say, logger):
100 |   """Triggered when a user asks for more results."""
101 |   ack()
102 |   state = body['state']
103 |   original_search = get_original_query_text(body['message'])
104 |   query_and_respond(say, original_search, state=state, num_results = 2)
105 | 
106 | @app.command("/vectara")
107 | def command_search(ack, respond, command):
108 |     ack()
109 |     text = command['text']
110 |     channel = None
111 |     user = None
112 |     parts = text.split(' ', 1)
113 |     if parts[0].startswith('<'):
114 |       channel_or_user = parts[0]
115 |       channel_or_user_parts = channel_or_user.split('|')
116 |       channel_or_user_id = channel_or_user_parts[0]
117 |       # channel is e.g. <#C03V4NCQJK2|foo-bar>.  can also be a person if it starts with @ e.g. <@C03V4NCQJK2|shane>
118 |       if channel_or_user_id.startswith('<@'):
119 |         user = get_user_filter(channel_or_user_id[2:])
120 |       elif channel_or_user_id.startswith('<#'):
121 |         channel = get_channel_filter(channel_or_user_id[2:])
122 |       text = parts[1]
123 |     query_and_respond(respond, search_text = text,
124 |                       filter_by_user=user, filter_by_channel=channel)
125 | 
126 | @app.event('app_home_opened')
127 | def home(client, event, logger):
128 |   client.views_publish(
129 |       # the user that opened your app's app home
130 |       user_id=event["user"],
131 |       # the view object that appears in the app home
132 |       view={
133 |         "type": "home",
134 |         "callback_id": "home_view",
135 | 
136 |         # body of the view
137 |         "blocks": homepage_blocks()
138 |       }
139 |     )
140 | 
141 | @app.event('message')
142 | def read_message(message, context, say):
143 |   """Triggered when a message is posted.
144 | 
145 |   This function looks for the bot's username, and if it's in the message, it
146 |   assumes the user wants to search and it extracts this as search text.
147 |   Otherwise, simply index the content
148 |   """
149 |   bot_user_id = context['bot_user_id']
150 | 
151 |   if 'text' in message:
152 |     message_text = message['text']
153 |     bot_user_id_reference = '<@{}>'.format(bot_user_id) #this is how a bot's mention shows up in the text
154 | 
155 |     if (message['channel_type'] == 'im'):
156 |       # this is a search request -- either a direct message to the bot or the bot is mentioned
157 |       search_text = message_text.replace(bot_user_id_reference,"").strip()
158 |       query_and_respond(say, search_text)
159 |     elif message['channel_type'] == 'channel':
160 |       if bot_user_id_reference in message_text:
161 |         search_text = message_text.replace(bot_user_id_reference,"").strip()
162 |         query_and_respond(say, search_text, filter_by_channel=get_channel_filter(message['channel']))
163 |       else:
164 |         #index the content
165 |         epoch_us = int(float(message['event_ts']) * 10000000)
166 |         link = 'https://{}.slack.com/archives/{}/p{}'.format(
167 |           os.environ.get('SLACK_WORKSPACE_SUBDOMAIN'),
168 |           message['channel'],
169 |           epoch_us
170 |         )
171 |         metadata = {
172 |           "message_link": link,
173 |           "message_type": message['type'],
174 |           "poster": message['user'],
175 |           "channel": message['channel'],
176 |           "channel_type": message['channel_type'],
177 |           "timestamp": float(message['event_ts'])
178 |         }
179 | 
180 |         index_message(
181 |           customer_id=int(os.environ.get('VECTARA_CUSTOMER_ID')),
182 |           corpus_id=int(os.environ.get('VECTARA_CORPUS_ID')),
183 |           text=message_text,
184 |           id=message['client_msg_id'],
185 |           title="Message from <@{}> at {}".format(message['user'], message['event_ts']),
186 |           metadata=metadata
187 |         )
188 |   else:
189 |     logging.error("Unhandled channel type: %s",message['channel_type'])
190 | 
191 | def query_and_respond(say, search_text = None, rerank = None,
192 |                       start_date = None, end_date = None, filter_by_user = None,
193 |                       filter_by_channel = None, num_results = 1):
194 |     """
195 |     """
196 |     if rerank == None and os.environ.get('VECTARA_USE_RERANKER') == 'true':
197 |       rerank = True
198 |     filters = [x for x in [filter_by_channel, filter_by_user, start_date, end_date] if x is not None]
199 |     
200 |     search_query, search_results = search(search_text=search_text,
201 |                                           rerank=rerank,
202 |                                           num_results=num_results,
203 |                                           metadata_filters=filters)
204 |     
205 |     if (len(search_results['responseSet']) > 0 and len(search_results['responseSet'][0]['response']) > 0):
206 |       # Always grab the last result, because if the user has asked for
207 |       # "more results", we set the search size and then look at the last result
208 |       response = search_results['responseSet'][0]['response'][-1]
209 |       text = response['text']
210 |       document = search_results['responseSet'][0]['document'][-1]
211 |       response_metadata = response['metadata']
212 |       document_metadata = document['metadata']
213 |       text = escape_markdown(text)
214 |       
215 |       # Grab the metadata from Vectara's response
216 |       link_meta = get_metadata_value(document_metadata, 'message_link')
217 |       poster = get_metadata_value(document_metadata, 'poster')
218 |       channel = get_metadata_value(document_metadata, 'channel')
219 |       timestamp = get_metadata_value(document_metadata, 'timestamp')
220 | 
221 |       # blocks are Slack's way of formatting messages.  For a reference, see
222 |       # https://app.slack.com/block-kit-builder/
223 |       blocks = [
224 |         {
225 |           "type": "section",
226 |           "text": {
227 |             "type": "mrkdwn",
228 |             "text": "Search results for: *{}*".format(search_text)
229 |           }
230 |         },
231 |         {
232 |           "type": "divider"
233 |         },
234 |         {
235 |           "type": "section",
236 |           "fields": [
237 |             {
238 |                 "type": "mrkdwn",
239 |                 "text": "<@{}> said:\n> {}".format(poster, text)
240 |             }
241 |           ]
242 |         },
243 |         {
244 |           "type": "section",
245 |           "text": { "type": "mrkdwn", "text": "<{}|Link>".format(link_meta) }
246 |         }
247 |       ]
248 | 
249 |       # Only add the reranker / more results, and filters if the user hasn't
250 |       # already used one of them.  This is an arbitrary limitation that just
251 |       # simplifies some logic and can be removed in the future
252 |       if len(filters) == 0:
253 |         blocks.append({
254 |             "type": "divider"
255 |         })
256 |         # Add our metadata filters
257 |         blocks.append({
258 |           "type": "section",
259 |           "text": { "type": "mrkdwn", "text": "User and Channel Filters:" }
260 |         })
261 |         blocks.append({
262 |           "type": "actions",
263 |           "elements": [
264 |             {
265 |               "type": "users_select",
266 |               "placeholder": {
267 |                 "type": "plain_text",
268 |                 "text": "Filter by user",
269 |                 "emoji": True
270 |               },
271 |               "action_id": "filter_by_user"
272 |             },
273 |             {
274 |               "type": "channels_select",
275 |               "placeholder": {
276 |                 "type": "plain_text",
277 |                 "text": "Filter by channel",
278 |                 "emoji": True
279 |               },
280 |               "action_id": "filter_by_channel"
281 |             }
282 |           ]
283 |         })
284 |         blocks.append({
285 |           "type": "section",
286 |           "text": { "type": "mrkdwn", "text": "Minimum and Maximum Post Time Filters:" }
287 |         })
288 |         blocks.append({
289 |           "type": "actions",
290 |           "elements": [
291 |             {
292 |               "type": "datepicker",
293 |               "initial_date": "2022-09-01",
294 |               "placeholder": {
295 |                 "type": "plain_text",
296 |                 "text": "Start Date",
297 |                 "emoji": True
298 |               },
299 |               "action_id": "filter_start_date"
300 |             },
301 |             {
302 |               "type": "datepicker",
303 |               "initial_date": (datetime.now() + timedelta(days=1)).strftime('%Y-%m-%d'),
304 |               "placeholder": {
305 |                 "type": "plain_text",
306 |                 "text": "End Date",
307 |                 "emoji": True
308 |               },
309 |               "action_id": "filter_end_date"
310 |             }
311 |           ]
312 |         })
313 |         blocks.insert(0, {
314 |           "type": "header",
315 |           "text": {
316 |             "type": "plain_text",
317 |             "text": "Search Results"
318 |           }
319 |         })
320 |       else:
321 |         if num_results != 1:
322 |           blocks.insert(0, {
323 |             "type": "header",
324 |             "text": {
325 |               "type": "plain_text",
326 |               "text": "Search Results (Result {})".format(num_results)
327 |             }
328 |           })
329 |       say(blocks=blocks, text="@{} said:\n> {}\n\n at {}".format(poster,text,timestamp), unfurl_links=False, unfurl_media=False)
330 |     else:
331 |       say("Sorry, I couldn't find any relevant results")
332 | 
333 | if __name__ == "__main__":
334 |     handler = SocketModeHandler(app, os.environ.get('SLACK_APP_TOKEN'))
335 |     handler.start()
336 | 


--------------------------------------------------------------------------------