5 |
6 | ## How to use
7 | - Streamlit can be used when you need to quickly expose a frontend for your Hyperplane service
8 | - To get started, replace the dummy function in `sent_infer_request_in_cluster` in skexample.py with your real request function
9 | - Copy the image_recognition folder to your github repo (and change it to the name that makes sense to your App)
10 | - Go to the service tab on the Hyperplane dashboard and start a service with the `streamlit_pipelin.yaml'
11 | - Choose the service URL prefix for your Frontend App, this will be the URL of your frontend. Assume it's `my_awesome_frontend`
12 | - After you hit the create button of the service and the service is up, you can then go to `https://yourdomain.hyperplane.dev/my_awesome_frontend` to see the frontend
13 | - Streamlit has python APIs to build frontend components such as boxes, buttons, radio inputs and so on. [Read more about Streamlit](https://docs.streamlit.io/)
14 | - Full documentation for streamlit please see the steamlit Documentation page.
15 |
--------------------------------------------------------------------------------
/example_notebooks/frontend/voila/multiple_notebooks/multiple_widgets.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "code",
5 | "execution_count": null,
6 | "id": "ebaa184b-765f-4261-9c5c-285bbde8fad5",
7 | "metadata": {},
8 | "outputs": [],
9 | "source": [
10 | "import ipywidgets as widgets\n",
11 | "def generate(n):\n",
12 | " widget = []\n",
13 | " for i in range(n):\n",
14 | " inner = []\n",
15 | " for j in range(n):\n",
16 | " inner.append(widgets.Button(description=f'{i*n+j+1}'))\n",
17 | " widget.append(widgets.HBox(inner))\n",
18 | "\n",
19 | " return widgets.VBox(widget)\n",
20 | "generate(20)"
21 | ]
22 | },
23 | {
24 | "cell_type": "code",
25 | "execution_count": null,
26 | "id": "8b59589a-7c71-4b1a-896a-b1139cbcf70f",
27 | "metadata": {},
28 | "outputs": [],
29 | "source": []
30 | }
31 | ],
32 | "metadata": {
33 | "kernelspec": {
34 | "display_name": "Python 3 (ipykernel)",
35 | "language": "python",
36 | "name": "python3"
37 | },
38 | "language_info": {
39 | "codemirror_mode": {
40 | "name": "ipython",
41 | "version": 3
42 | },
43 | "file_extension": ".py",
44 | "mimetype": "text/x-python",
45 | "name": "python",
46 | "nbconvert_exporter": "python",
47 | "pygments_lexer": "ipython3",
48 | "version": "3.9.6"
49 | }
50 | },
51 | "nbformat": 4,
52 | "nbformat_minor": 5
53 | }
54 |
--------------------------------------------------------------------------------
/LLMs/QA_app/Readme.md:
--------------------------------------------------------------------------------
1 | # Building a PDF Knowledge Bot with Open-source LLMs on Shakudo
2 |
3 | ## Solution Overview:
4 | For any textual knowledge base (in our case, PDFs), we first need to extract text snippets from the knowledge base and use an embedding model to create a vector store representing the semantic content of the snippets. When a question is asked, we estimate its embedding and find relevant snippets using an efficient similarity search from vector stores. After extracting the snippets, we engineer a prompt and generate an answer using the LLM generation model. The prompt can be tuned based on the specific LLM used.
5 |
6 | 
7 | Experimentation and development are crucial elements in the field of data science. Shakudo's session facilitates the selection of the appropriate computing resources and provides the flexibility to choose Jupyter Notebooks, VS Code Server (provided by the platform) or connecting via SSH to use a preferred local editor.
8 |
9 |
10 | * Jupyter notebook for tutorial : [pdf_qa](./pdf_qa.ipynb)
11 | * Associated blog tutorial: [Building a PDF Knowledge Bot With Open-Source LLMs- A Step-by-Step Guide
12 | ](https://www.shakudo.io/blog)
13 |
14 |
15 | ### Code References:
16 | * The code is adapted based on the work in [LLM-WikipediaQA](https://github.com/georgesung/LLM-WikipediaQA/tree/main), where the author compares FastChat-T5, Flan-T5 with ChatGPT running a Q&A on Wikipedia Articles
17 |
18 |
--------------------------------------------------------------------------------
/example_notebooks/frontend/voila/multiple_notebooks/bqplot.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "metadata": {},
6 | "source": [
7 | "# So easy, *voilà*!\n",
8 | "\n",
9 | "In this example notebook, we demonstrate how Voilà can render custom Jupyter widgets such as [bqplot](https://github.com/bloomberg/bqplot). "
10 | ]
11 | },
12 | {
13 | "cell_type": "code",
14 | "execution_count": null,
15 | "metadata": {},
16 | "outputs": [],
17 | "source": [
18 | "import warnings\n",
19 | "warnings.filterwarnings('ignore')"
20 | ]
21 | },
22 | {
23 | "cell_type": "code",
24 | "execution_count": null,
25 | "metadata": {},
26 | "outputs": [],
27 | "source": [
28 | "import numpy as np\n",
29 | "from bqplot import pyplot as plt\n",
30 | "\n",
31 | "plt.figure(1, title='Line Chart')\n",
32 | "np.random.seed(0)\n",
33 | "n = 200\n",
34 | "x = np.linspace(0.0, 10.0, n)\n",
35 | "y = np.cumsum(np.random.randn(n))\n",
36 | "plt.plot(x, y)\n",
37 | "plt.show()"
38 | ]
39 | }
40 | ],
41 | "metadata": {
42 | "kernelspec": {
43 | "display_name": "Python 3",
44 | "language": "python",
45 | "name": "python3"
46 | },
47 | "language_info": {
48 | "codemirror_mode": {
49 | "name": "ipython",
50 | "version": 3
51 | },
52 | "file_extension": ".py",
53 | "mimetype": "text/x-python",
54 | "name": "python",
55 | "nbconvert_exporter": "python",
56 | "pygments_lexer": "ipython3",
57 | "version": "3.7.3"
58 | }
59 | },
60 | "nbformat": 4,
61 | "nbformat_minor": 2
62 | }
63 |
--------------------------------------------------------------------------------
/example_notebooks/frontend/voila/multiple_notebooks/interactive.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "metadata": {},
6 | "source": [
7 | "# So easy, *voilà*!\n",
8 | "\n",
9 | "In this example notebook, we demonstrate how Voilà can render notebooks making use of ipywidget's `@interact`."
10 | ]
11 | },
12 | {
13 | "cell_type": "code",
14 | "execution_count": null,
15 | "metadata": {},
16 | "outputs": [],
17 | "source": [
18 | "from ipywidgets import HBox, VBox, IntSlider, interactive_output\n",
19 | "from IPython.display import display\n",
20 | "\n",
21 | "a = IntSlider()\n",
22 | "b = IntSlider()\n",
23 | "\n",
24 | "def f(a, b):\n",
25 | " print(\"{} * {} = {}\".format(a, b, a * b))\n",
26 | "\n",
27 | "out = interactive_output(f, { \"a\": a, \"b\": b })\n",
28 | "\n",
29 | "display(HBox([VBox([a, b]), out]))"
30 | ]
31 | },
32 | {
33 | "cell_type": "code",
34 | "execution_count": null,
35 | "metadata": {},
36 | "outputs": [],
37 | "source": []
38 | }
39 | ],
40 | "metadata": {
41 | "kernelspec": {
42 | "display_name": "Python 3",
43 | "language": "python",
44 | "name": "python3"
45 | },
46 | "language_info": {
47 | "codemirror_mode": {
48 | "name": "ipython",
49 | "version": 3
50 | },
51 | "file_extension": ".py",
52 | "mimetype": "text/x-python",
53 | "name": "python",
54 | "nbconvert_exporter": "python",
55 | "pygments_lexer": "ipython3",
56 | "version": "3.7.3"
57 | }
58 | },
59 | "nbformat": 4,
60 | "nbformat_minor": 2
61 | }
62 |
--------------------------------------------------------------------------------
/example_notebooks/serving/IoT-detection/utils.py:
--------------------------------------------------------------------------------
1 | """
2 | helper function
3 |
4 | """
5 | def download_from_cloud(local_file_name, remote_file_name):
6 | """
7 | Upload a file to gcs bucket or s3 bucket.
8 | """
9 | import os
10 |
11 | cloud_name = remote_file_name.split('://')[0]
12 | if cloud_name =='gs':
13 | import gcsfs
14 | fs = gcsfs.GCSFileSystem(project=os.environ['GCP_PROJECT'])
15 | elif cloud_name =='s3':
16 | import s3fs
17 | fs = s3fs.S3FileSystem()
18 | else:
19 | raise NameError(f'cloud name {cloud_name} unknown')
20 | try:
21 | print(f'Downloading from {remote_file_name} to {local_file_name}')
22 | fs.get(remote_file_name, local_file_name)
23 | print("done downloading!")
24 | except Exception as exp:
25 | print(f"Download failed: {exp}")
26 | return
27 |
28 | def upload_to_cloud(local_file_name, remote_file_name):
29 | """
30 | Upload a file to gcs bucket or s3 bucket.
31 | """
32 | import os
33 | cloud_name = remote_file_name.split('://')[0]
34 | if cloud_name =='gs':
35 | import gcsfs
36 | fs = gcsfs.GCSFileSystem(project=os.environ['GCP_PROJECT'])
37 | elif cloud_name =='s3':
38 | import s3fs
39 | fs = s3fs.S3FileSystem()
40 | else:
41 | raise NameError(f'cloud name {cloud_name} unknown')
42 | try:
43 | print(f'Uploading from {local_file_name} to {remote_file_name}')
44 | fs.put(local_file_name, remote_file_name)
45 | print("done uploading!")
46 | except Exception as exp:
47 | print(f"Uploading failed: {exp}")
48 |
49 | return fs
--------------------------------------------------------------------------------
/example_notebooks/frontend/streamlit_app_example/image_recognition/slexample.py:
--------------------------------------------------------------------------------
1 | # frontend/main.py
2 |
3 | import requests
4 | import streamlit as st
5 | from PIL import Image
6 | import json
7 | import numpy as np
8 | import pandas as pd
9 | import base64
10 | import argparse
11 | import time
12 | import random
13 |
14 |
15 | def sent_infer_request_in_cluster(img):
16 | ## send request to a model inference endpoint that's served on hyperplane
17 | ## As the services are all internally authenticated, no need to pass JWT tokens
18 | endpoint = "http://image_recognition-inference-endpoint:8787/infer"
19 | data = {
20 | "image": img
21 | }
22 | headers = {
23 | "Content-Type": "application/json",
24 | }
25 |
26 | # result = requests.post(endpoint, data = json.dumps(data), headers=headers).json()
27 |
28 | ## for demo purpose we are going to use a random result
29 | try:
30 | result = {"category": random.choice(["cat", "cat", "cat"])}
31 | print(result)
32 | except:
33 | result = {}
34 |
35 | return result
36 |
37 |
38 | ##
39 | ## Streamlit UI components
40 | ##
41 |
42 |
43 | st.set_option("deprecation.showfileUploaderEncoding", False)
44 |
45 | # defines an h1 header
46 | st.title("Image Recognition Cats Dogs Fruits")
47 |
48 | # displays a file uploader widget
49 | image_upload = st.file_uploader("Choose an image")
50 |
51 | # displays a file uploader widget
52 | image_type = st.selectbox("Choose an image type", ['fruit', 'animal', 'other'])
53 |
54 | # displays a button
55 | if st.button("Get annotation"):
56 | if image_upload is not None:
57 | # try:
58 | image_raw = Image.open(image_upload)
59 | image = np.array(image_raw)[:, :, ::-1]
60 |
61 | result = sent_infer_request_in_cluster(image)
62 | st.json(result)
63 | # except:
64 | # st.text("Could not process image")
65 |
66 |
67 | image = Image.open('cat.png')
68 |
69 | st.image(image, caption='Cat')
70 |
71 |
--------------------------------------------------------------------------------
/example_notebooks/frontend/sentence_classification/slnlp.py:
--------------------------------------------------------------------------------
1 | import requests
2 | import streamlit as st
3 | import pandas as pd
4 | import json
5 | from io import StringIO
6 |
7 | def run_sentiment_analysis(string):
8 | ## send request
9 | endpoint = "your_model_end_point:8787/"
10 | data = {
11 | "string": string
12 | }
13 | headers = {
14 | "Content-Type": "application/json",
15 | }
16 |
17 | try:
18 | result = requests.post(endpoint, data = json.dumps(data), headers=headers).json()
19 | # print('result obtained')
20 | # print(result)
21 | except requests.exceptions.HTTPError as err:
22 | raise SystemExit(err)
23 |
24 | return result
25 |
26 |
27 |
28 | # defines an h1 header
29 | st.title("Sentence analysis")
30 |
31 | st.subheader('Paste some text here get classified')
32 |
33 | ## type text in the box
34 | txt = st.text_area('Text to analyze', '''
35 | ''')
36 |
37 | st.write("Sentence:", txt)
38 |
39 | txt_result = run_sentiment_analysis(txt)
40 | # print('txt_result', txt_result)
41 |
42 | # st.write('Topic:', txt_result)
43 | st.metric(label="Topic", value=txt_result)
44 |
45 | st.subheader('Upload a text file here to get classified')
46 |
47 | ## file uploader
48 | uploaded_file = st.file_uploader("Choose a file")
49 | if uploaded_file is not None:
50 | # To read file as bytes:
51 | bytes_data = uploaded_file.getvalue()
52 | # st.write(bytes_data)
53 |
54 | # # To convert to a string based IO:
55 | stringio = StringIO(uploaded_file.getvalue().decode("utf-8"))
56 | # st.write(stringio)
57 |
58 | # # To read file as string:
59 | string_data = stringio.read()
60 | # st.write(string_data)
61 |
62 |
63 | # Can be used wherever a "file-like" object is accepted:
64 | dataframe = pd.read_csv(uploaded_file)
65 | # print(dataframe)
66 | results = []
67 | for sentence in dataframe.Sentence.tolist():
68 | results.append(run_sentiment_analysis(sentence))
69 | dataframe['Topic Prediction'] = results
70 | st.write(dataframe)
--------------------------------------------------------------------------------
/example_notebooks/frontend/voila/multiple_notebooks/reveal.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "code",
5 | "execution_count": null,
6 | "metadata": {
7 | "slideshow": {
8 | "slide_type": "slide"
9 | }
10 | },
11 | "outputs": [],
12 | "source": [
13 | "print('hi')"
14 | ]
15 | },
16 | {
17 | "cell_type": "code",
18 | "execution_count": null,
19 | "metadata": {},
20 | "outputs": [],
21 | "source": [
22 | "import ipywidgets as widgets\n",
23 | "slider = widgets.FloatSlider(description='x')\n",
24 | "text = widgets.FloatText(disabled=True, description='$x^2$')\n",
25 | "text.disabled\n",
26 | "def compute(*ignore):\n",
27 | " text.value = str(slider.value**2)\n",
28 | "slider.observe(compute, 'value')\n",
29 | "slider.value = 14\n",
30 | "widgets.VBox([slider, text])"
31 | ]
32 | },
33 | {
34 | "cell_type": "code",
35 | "execution_count": null,
36 | "metadata": {
37 | "slideshow": {
38 | "slide_type": "slide"
39 | }
40 | },
41 | "outputs": [],
42 | "source": [
43 | "print('voila')"
44 | ]
45 | },
46 | {
47 | "cell_type": "code",
48 | "execution_count": null,
49 | "metadata": {
50 | "slideshow": {
51 | "slide_type": "subslide"
52 | }
53 | },
54 | "outputs": [],
55 | "source": [
56 | "1+2"
57 | ]
58 | },
59 | {
60 | "cell_type": "code",
61 | "execution_count": null,
62 | "metadata": {},
63 | "outputs": [],
64 | "source": []
65 | }
66 | ],
67 | "metadata": {
68 | "celltoolbar": "Slideshow",
69 | "kernelspec": {
70 | "display_name": "Python 3 (ipykernel)",
71 | "language": "python",
72 | "name": "python3"
73 | },
74 | "language_info": {
75 | "codemirror_mode": {
76 | "name": "ipython",
77 | "version": 3
78 | },
79 | "file_extension": ".py",
80 | "mimetype": "text/x-python",
81 | "name": "python",
82 | "nbconvert_exporter": "python",
83 | "pygments_lexer": "ipython3",
84 | "version": "3.8.10"
85 | },
86 | "voila": {
87 | "template": "reveal"
88 | }
89 | },
90 | "nbformat": 4,
91 | "nbformat_minor": 4
92 | }
93 |
--------------------------------------------------------------------------------
/example_notebooks/inference/batch_inference/scheduler_setup.py:
--------------------------------------------------------------------------------
1 | """
2 | startup script to be preloaded to the workers
3 |
4 | """
5 | import pip._internal as pip
6 | def install(package):
7 | pip.main(['install', package])
8 |
9 | try:
10 | import s3fs
11 | except ImportError:
12 | install('s3fs')
13 | import s3fs
14 |
15 | import gcsfs
16 | import os
17 | import pickle
18 | from transformers import DistilBertTokenizer, TFDistilBertForSequenceClassification
19 |
20 |
21 |
22 | ## start up script
23 |
24 | def download_from_cloud(local_file_name, remote_file_name):
25 | """
26 | Download a file to gcp or s3.
27 | """
28 | import os
29 | import s3fs
30 | import gcsfs
31 | cloud_name = remote_file_name.split('://')[0]
32 | if cloud_name =='gs':
33 | fs = gcsfs.GCSFileSystem(project=os.environ['GCP_PROJECT'])
34 | elif cloud_name =='s3':
35 | fs = s3fs.S3FileSystem()
36 | else:
37 | raise NameError(f'cloud name {cloud_name} unknown')
38 | try:
39 | print(f'download from {remote_file_name} to {local_file_name}')
40 | fs.get(remote_file_name, local_file_name, recursive=True)
41 | print("done downloading!")
42 | except Exception as exp:
43 | print(f"download failed: {exp}")
44 |
45 | return
46 |
47 |
48 | def load_models(model_path):
49 | """
50 | Load the model from the unzipped local model path
51 | """
52 |
53 | model = TFDistilBertForSequenceClassification.from_pretrained(f'{model_path}/clf')
54 | model_name, max_len = pickle.load(open(f'{model_path}/info.pkl', 'rb'))
55 | loaded_models['model'] = (model, model_name, max_len)
56 |
57 | return loaded_models
58 |
59 |
60 | def load_model_from_pretrained(model_name):
61 | """
62 | Load the model from the unzipped local model path
63 | """
64 |
65 | model = TFDistilBertForSequenceClassification.from_pretrained(model_name)
66 | max_len = 20
67 | loaded_models['model'] = (model, model_name, max_len)
68 |
69 | return loaded_models
70 |
71 | loaded_models = {}
72 | model_name = "distilbert-base-uncased"
73 | loaded_models = load_model_from_pretrained(model_name)
74 |
--------------------------------------------------------------------------------
/example_notebooks/frontend/voila/multiple_notebooks/basics.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "source": [
6 | "# So easy, *voilà*!\n",
7 | "\n",
8 | "In this example notebook, we demonstrate how Voilà can render Jupyter notebooks with interactions requiring a roundtrip to the kernel."
9 | ],
10 | "metadata": {}
11 | },
12 | {
13 | "cell_type": "markdown",
14 | "source": [
15 | "## Jupyter Widgets"
16 | ],
17 | "metadata": {}
18 | },
19 | {
20 | "cell_type": "code",
21 | "execution_count": null,
22 | "source": [
23 | "import ipywidgets as widgets\n",
24 | "\n",
25 | "slider = widgets.FloatSlider(description='$x$')\n",
26 | "text = widgets.FloatText(disabled=True, description='$x^2$')\n",
27 | "\n",
28 | "def compute(*ignore):\n",
29 | " text.value = str(slider.value ** 2)\n",
30 | "\n",
31 | "slider.observe(compute, 'value')\n",
32 | "\n",
33 | "slider.value = 4\n",
34 | "\n",
35 | "widgets.VBox([slider, text])"
36 | ],
37 | "outputs": [],
38 | "metadata": {}
39 | },
40 | {
41 | "cell_type": "markdown",
42 | "source": [
43 | "## Basic outputs of code cells"
44 | ],
45 | "metadata": {}
46 | },
47 | {
48 | "cell_type": "code",
49 | "execution_count": null,
50 | "source": [
51 | "import pandas as pd\n",
52 | "\n",
53 | "iris = pd.read_csv('https://raw.githubusercontent.com/mwaskom/seaborn-data/master/iris.csv')\n",
54 | "iris"
55 | ],
56 | "outputs": [],
57 | "metadata": {}
58 | },
59 | {
60 | "cell_type": "code",
61 | "execution_count": null,
62 | "source": [],
63 | "outputs": [],
64 | "metadata": {}
65 | }
66 | ],
67 | "metadata": {
68 | "kernelspec": {
69 | "display_name": "Python 3",
70 | "language": "python",
71 | "name": "python3"
72 | },
73 | "language_info": {
74 | "codemirror_mode": {
75 | "name": "ipython",
76 | "version": 3
77 | },
78 | "file_extension": ".py",
79 | "mimetype": "text/x-python",
80 | "name": "python",
81 | "nbconvert_exporter": "python",
82 | "pygments_lexer": "ipython3",
83 | "version": "3.8.5"
84 | }
85 | },
86 | "nbformat": 4,
87 | "nbformat_minor": 4
88 | }
--------------------------------------------------------------------------------
/example_notebooks/serving/IoT-detection/main.py:
--------------------------------------------------------------------------------
1 | """
2 | ### simple FastAPI App for inference with the trained AutoML model
3 | """
4 |
5 | from fastapi import FastAPI, Request
6 | from fastapi.encoders import jsonable_encoder
7 | from pydantic import BaseModel
8 | import time
9 | import pandas as pd
10 | import numpy as np
11 | import json
12 | import io
13 | import os
14 | import sys
15 | from autogluon.tabular import TabularDataset, TabularPredictor
16 | from utils import upload_to_cloud, download_from_cloud
17 |
18 |
19 | def download_model():
20 | try:
21 | local_file_path = 'models'
22 | remote_file_path =f"gs://shakdemo-hyperplane/data/environmental-sensor-data/{local_file_path}"
23 | download_from_cloud(local_file_path, remote_file_path)
24 | print('downloaded model successfully')
25 | except Exception as e:
26 | print(f'download failed with {e}')
27 | return
28 |
29 |
30 | def run_inference(data:dict) ->pd.DataFrame:
31 | df = pd.DataFrame.from_dict(data, orient = 'index').T
32 | data = TabularDataset(df)
33 | predictor = TabularPredictor.load('models')
34 | y_pred = predictor.predict(df)
35 | return y_pred
36 |
37 |
38 | global model
39 | model = download_model()
40 |
41 | class MyRequest(BaseModel):
42 | data: dict
43 |
44 | app = FastAPI()
45 |
46 |
47 | @app.get("/health-ready")
48 | def health_check():
49 | return jsonable_encoder({"message": "Ready"})
50 |
51 | @app.get("/")
52 | def root():
53 | return {"hello":"world"}
54 |
55 | @app.post("/infer")
56 | async def infer(req: Request):
57 | if req.headers['Content-Type'] == 'application/json':
58 | i = MyRequest(** await req.json())
59 | elif req.headers['Content-Type'] == 'multipart/form-data':
60 | i = MyRequest(** await req.form())
61 | elif req.headers['Content-Type'] == 'application/x-www-form-urlencoded':
62 | i = MyRequest(** await req.form())
63 | r = json.loads(i.json())
64 | return {
65 | 'prediction': str(run_inference(data = r['data']).values[0])
66 | }
67 |
68 |
69 | if __name__ == '__main__':
70 | jsondata = {}
71 | data = {
72 | 'ts': 1594512094.3859746,
73 | 'co': 0.0049559386483912,
74 | 'humidity': 51.0,
75 | 'light': 0.0,
76 | 'lpg': 0.0076508222705571,
77 | 'smoke': 0.0204112701224129,
78 | 'temp': 22.7
79 | }
80 | jsondata['data'] = data
81 | jsondata = json.dumps(jsondata)
82 | print(jsondata)
83 | starttime = time.time()
84 | r = json.loads(jsondata)
85 | output = run_inference(r['data'])
86 | print(output)
87 | print(f'used {time.time() - starttime} seconds')
88 |
--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
1 | # Byte-compiled / optimized / DLL files
2 | __pycache__/
3 | *.py[cod]
4 | *$py.class
5 |
6 | # C extensions
7 | *.so
8 |
9 | # Distribution / packaging
10 | .Python
11 | build/
12 | develop-eggs/
13 | dist/
14 | downloads/
15 | eggs/
16 | .eggs/
17 | lib/
18 | lib64/
19 | parts/
20 | sdist/
21 | var/
22 | wheels/
23 | pip-wheel-metadata/
24 | share/python-wheels/
25 | *.egg-info/
26 | .installed.cfg
27 | *.egg
28 | MANIFEST
29 |
30 | # PyInstaller
31 | # Usually these files are written by a python script from a template
32 | # before PyInstaller builds the exe, so as to inject date/other infos into it.
33 | *.manifest
34 | *.spec
35 |
36 | # Installer logs
37 | pip-log.txt
38 | pip-delete-this-directory.txt
39 |
40 | # Unit test / coverage reports
41 | htmlcov/
42 | .tox/
43 | .nox/
44 | .coverage
45 | .coverage.*
46 | .cache
47 | nosetests.xml
48 | coverage.xml
49 | *.cover
50 | *.py,cover
51 | .hypothesis/
52 | .pytest_cache/
53 |
54 | # Translations
55 | *.mo
56 | *.pot
57 |
58 | # Django stuff:
59 | *.log
60 | local_settings.py
61 | db.sqlite3
62 | db.sqlite3-journal
63 |
64 | # Flask stuff:
65 | instance/
66 | .webassets-cache
67 |
68 | # Scrapy stuff:
69 | .scrapy
70 |
71 | # Sphinx documentation
72 | docs/_build/
73 |
74 | # PyBuilder
75 | target/
76 |
77 | # Jupyter Notebook
78 | .ipynb_checkpoints
79 |
80 | # IPython
81 | profile_default/
82 | ipython_config.py
83 |
84 | # pyenv
85 | .python-version
86 |
87 | # pipenv
88 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
89 | # However, in case of collaboration, if having platform-specific dependencies or dependencies
90 | # having no cross-platform support, pipenv may install dependencies that don't work, or not
91 | # install all needed dependencies.
92 | #Pipfile.lock
93 |
94 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow
95 | __pypackages__/
96 |
97 | # Celery stuff
98 | celerybeat-schedule
99 | celerybeat.pid
100 |
101 | # SageMath parsed files
102 | *.sage.py
103 |
104 | # Environments
105 | .env
106 | .venv
107 | env/
108 | venv/
109 | ENV/
110 | env.bak/
111 | venv.bak/
112 |
113 | # Spyder project settings
114 | .spyderproject
115 | .spyproject
116 |
117 | # Rope project settings
118 | .ropeproject
119 |
120 | # mkdocs documentation
121 | /site
122 |
123 | # mypy
124 | .mypy_cache/
125 | .dmypy.json
126 | dmypy.json
127 |
128 | # Pyre type checker
129 | .pyre/
130 | LLMs/confluence_app/=0.0.19
131 | LLMs/confluence_app/.chroma/*
132 | LLMs/confluence_app/db/*
133 | LLMs/confluence_app/chroma_db/*
134 |
135 | ## Venvs in the folder
136 | *.venv*
137 |
--------------------------------------------------------------------------------
/example_notebooks/serving/triton/sentence_classification_app/clients/sentence_classification.py:
--------------------------------------------------------------------------------
1 | import argparse
2 | import numpy as np
3 | import sys
4 | from functools import partial
5 | import os
6 | import tritongrpcclient
7 | import tritongrpcclient.model_config_pb2 as mc
8 | import tritonhttpclient
9 | from tritonclientutils import triton_to_np_dtype
10 | from tritonclientutils import InferenceServerException
11 | from transformers import XLMRobertaTokenizer
12 | from scipy.special import softmax
13 | R_tokenizer = XLMRobertaTokenizer.from_pretrained('joeddav/xlm-roberta-large-xnli')
14 | VERBOSE = False
15 | # hypothesis for topic classification
16 | topic = 'This text is about space & cosmos'
17 | input_name = ['input__0', 'input__1']
18 | output_name = 'output__0'
19 | def run_inference(premise, model_name='deepset', url='hyperplane-triton.default:8000', model_version='1'):
20 | triton_client = tritonhttpclient.InferenceServerClient(
21 | url=url, verbose=VERBOSE)
22 | model_metadata = triton_client.get_model_metadata(
23 | model_name=model_name, model_version=model_version)
24 | model_config = triton_client.get_model_config(
25 | model_name=model_name, model_version=model_version)
26 | # I have restricted the input sequence length to 256
27 | input_ids = R_tokenizer.encode(premise, topic, max_length=256, truncation=True, padding='max_length')
28 | input_ids = np.array(input_ids, dtype=np.int32)
29 | mask = input_ids != 1
30 | mask = np.array(mask, dtype=np.int32)
31 |
32 | mask = mask.reshape(1, 256)
33 | input_ids = input_ids.reshape(1, 256)
34 | input0 = tritonhttpclient.InferInput(input_name[0], (1, 256), 'INT32')
35 | input0.set_data_from_numpy(input_ids, binary_data=False)
36 | input1 = tritonhttpclient.InferInput(input_name[1], (1, 256), 'INT32')
37 | input1.set_data_from_numpy(mask, binary_data=False)
38 | output = tritonhttpclient.InferRequestedOutput(output_name, binary_data=False)
39 | response = triton_client.infer(model_name, model_version=model_version, inputs=[input0, input1], outputs=[output])
40 | logits = response.as_numpy('output__0')
41 | logits = np.asarray(logits, dtype=np.float32)
42 | # we throw away "neutral" (dim 1) and take the probability of
43 | # "entailment" (2) as the probability of the label being true
44 | entail_contradiction_logits = logits[:,[0,2]]
45 | probs = softmax(entail_contradiction_logits)
46 | true_prob = probs[:,1].item() * 100
47 | print(f'Probability that the label is true: {true_prob:0.2f}%')
48 | # topic classification premises
49 | if __name__ == '__main__':
50 | run_inference('Jupiter’s Biggest Moons Started as Tiny Grains of Hail')
51 |
52 |
--------------------------------------------------------------------------------
/example_notebooks/serving/triton/hyperplane-triton-api/app.py:
--------------------------------------------------------------------------------
1 | import os
2 | import json
3 | from flask import request
4 | from flask import jsonify
5 | from datetime import datetime
6 |
7 | from geventhttpclient import client
8 | from clients.image_client import run
9 |
10 | from flask import Flask, Response, render_template
11 |
12 | application = Flask(__name__)
13 |
14 | def get_boolean(val, default):
15 | try:
16 | if(str(val).lower() == "false"):
17 | val = False
18 | elif(str(val).lower() == "true"):
19 | val = True
20 | except Exception as e:
21 | print(e)
22 | return default
23 |
24 | def get_number(val, default):
25 | try:
26 | n = int(val)
27 | return n
28 | except Exception as e:
29 | return default
30 |
31 | @application.route("/health-ready", methods=['GET'])
32 | def health_check():
33 | return jsonify({"message": "Ready"})
34 |
35 | @application.route("/", methods=['POST'], strict_slashes=False)
36 | def image_client():
37 | try:
38 | data = request.get_json()
39 | if(data != None):
40 | image = data.get("image", "")
41 | model_name = data.get("model_name", "")
42 | url = data.get("url", "hyperplane-triton.default:8000")
43 | elif(request.form != None):
44 | image = request.form.to_dict()["image"]
45 | model_name = request.form.to_dict()["model_name"]
46 | url = request.form.to_dict()["url"]
47 | else:
48 | data = request.get_json(force=True)
49 | image = data.get("image", "")
50 | model_name = data.get("model_name", "")
51 | url = data.get("url", "hyperplane-triton.default:8000")
52 |
53 | client_run = run(
54 | image = image,
55 | model_name = model_name,
56 | verbose = False,
57 | async_set = False,
58 | streaming = False,
59 | model_version = "",
60 | batch_size= 1,
61 | classes = 3,
62 | scaling = "INCEPTION",
63 | url = url,
64 | protocol = "http"
65 | )
66 | response = application.response_class(
67 | response=client_run,
68 | status=200,
69 | mimetype='application/json'
70 | )
71 | return response
72 | except Exception as e:
73 | print(e)
74 | response = json.dumps({
75 | "hasError" : True,
76 | "error": "There was an error processing your request. Please check your inputs",
77 | "errorMessage": str(e)
78 | })
79 | return jsonify(response)
80 |
81 |
82 | if __name__ == "__main__":
83 | application.run(debug=True)
84 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # Example notebooks
2 | You can find here a list of the official notebooks and scripts provided by Hyperplane.
3 |
4 | Also, we would like to list here interesting content created by the community. If you wrote some notebook(s) leveraging Hyperplane and would like be listed here, please open a Pull Request so it can be included under the Community notebooks.
5 |
6 | | Notebook | Description |
7 | | :----------------| :--------------------------------------------------------------------------- |
8 | | [Data processing with DASK](https://github.com/devsentient/examples/blob/main/example_notebooks/data_prep/dask_group_sort.ipynb) | Speed up data preprocessing with distributed DASK cluster on Hyperplane |
9 | | [Data prepcessing with Ray](https://github.com/devsentient/examples/blob/main/example_notebooks/data_prep/ray_data_processing.ipynb) | speed up data preprocessing with distributed Ray cluster on Hyperplane |
10 | | [Data preprocessing with Spark](https://github.com/devsentient/examples/blob/main/example_notebooks/data_prep/spark_on_ray_data_processing.ipynb) | speed up data preprocessing with distributed spark on Ray clusters on Hyperplane |
11 | | [Question answering Tensorflow Training with Ray ](https://github.com/devsentient/examples/blob/main/example_notebooks/training/ray_tf_nlp.ipynb) | Tensorflow training with distributed hyperparameter tuning on Ray cluster |
12 | | [CIFAR Pytorch Training with Ray ](https://github.com/devsentient/examples/blob/main/example_notebooks/training/ray_training_torch.ipynb) | pytorch training with distributed hyperparameter tuning and MLFlow on Ray cluster |
13 | | [Ray Tune with MLFlow](https://github.com/devsentient/examples/blob/main/example_notebooks/training/ray_mlflow.ipynb) | Simple Ray Tune example wth MLFlow Tracking |
14 | | [Ray Tune Bayesian Optimization](https://github.com/devsentient/examples/blob/main/example_notebooks/training/ray_tune_demo.ipynb) | A collection of Ray Tune scheduler examples |
15 | | [Speed up inference on large data with DASK ](https://github.com/devsentient/examples/blob/main/example_notebooks/inference/batch_inference) | Advance example on speed inference with DASK by preload large model to DASK workers |
16 | | [Triton model prepration ](https://github.com/devsentient/examples/tree/main/example_notebooks/serving/triton) | Convert Pytorch Keras sklearn and xgboost model checkpoints for Triton serving |
17 | | [Hyperplane Triton service App ](https://github.com/devsentient/examples/tree/main/example_notebooks/serving/triton) | Triton client in a flask APP to be served as a Hyperplane service|
18 | | [Simple pipeline job ](https://github.com/devsentient/examples/tree/main/example_notebooks/pipelines/a_simple_pipelinen) | A basic pipeline to automate a jupyter notebook with parameterization |
19 | | [GraphQL within jupyter ](https://github.com/devsentient/examples/blob/main/example_notebooks/utils/submit_graphql_with_python.ipynb) | Submit pipeline jobs using graphql queries within jupyter a notebook
20 |
--------------------------------------------------------------------------------
/example_notebooks/frontend/voila/multiple_notebooks/gridspecLayout.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "metadata": {},
6 | "source": [
7 | "# Which multiplication table do you want to learn?\n",
8 | "\n",
9 | "In this example notebook we demonstrate how Voilà can render different Jupyter widgets using [GridspecLayout](https://ipywidgets.readthedocs.io/en/latest/examples/Layout%20Templates.html#Grid-layout)"
10 | ]
11 | },
12 | {
13 | "cell_type": "code",
14 | "execution_count": null,
15 | "metadata": {
16 | "scrolled": false
17 | },
18 | "outputs": [],
19 | "source": [
20 | "from ipywidgets import GridspecLayout, Button, BoundedIntText, Valid, Layout, Dropdown\n",
21 | "\n",
22 | "def create_expanded_button(description, button_style):\n",
23 | " return Button(description=description, button_style=button_style, layout=Layout(height='auto', width='auto'))\n",
24 | " \n",
25 | "rows = 11\n",
26 | "columns = 6\n",
27 | "\n",
28 | "gs = GridspecLayout(rows, columns)\n",
29 | "\n",
30 | "def on_result_change(change):\n",
31 | " row = int(change[\"owner\"].layout.grid_row)\n",
32 | " gs[row, 5].value = gs[0, 0].value * row == change[\"new\"]\n",
33 | " \n",
34 | "def on_multipler_change(change):\n",
35 | " for i in range(1, rows):\n",
36 | " gs[i, 0].description = str(change[\"new\"])\n",
37 | " gs[i, 4].max = change[\"new\"] * 10\n",
38 | " gs[i, 4].value = 1\n",
39 | " gs[i, 4].step = change[\"new\"]\n",
40 | " gs[i, 5].value = False\n",
41 | "\n",
42 | "gs[0, 0] = Dropdown(\n",
43 | " options=[1, 2, 3, 4, 5, 6, 7, 8, 9, 10],\n",
44 | " value=2,\n",
45 | ")\n",
46 | "gs[0, 0].observe(on_multipler_change, names=\"value\")\n",
47 | "multiplier = gs[0, 0].value\n",
48 | "\n",
49 | "for i in range(1, rows):\n",
50 | " gs[i, 0] = create_expanded_button(str(multiplier), \"\")\n",
51 | " gs[i, 1] = create_expanded_button(\"*\", \"\")\n",
52 | " gs[i, 2] = create_expanded_button(str(i), \"info\")\n",
53 | " gs[i, 3] = create_expanded_button(\"=\", \"\")\n",
54 | "\n",
55 | " gs[i, 4] = BoundedIntText(\n",
56 | " min=0,\n",
57 | " max=multiplier * 10,\n",
58 | " layout=Layout(grid_row=str(i)),\n",
59 | " value=1,\n",
60 | " step=multiplier,\n",
61 | " disabled=False\n",
62 | " )\n",
63 | "\n",
64 | " gs[i, 5] = Valid(\n",
65 | " value=False,\n",
66 | " description='Valid!',\n",
67 | " )\n",
68 | "\n",
69 | " gs[i, 4].observe(on_result_change, names='value')\n",
70 | "\n",
71 | "gs"
72 | ]
73 | }
74 | ],
75 | "metadata": {
76 | "kernelspec": {
77 | "display_name": "Python 3",
78 | "language": "python",
79 | "name": "python3"
80 | },
81 | "language_info": {
82 | "codemirror_mode": {
83 | "name": "ipython",
84 | "version": 3
85 | },
86 | "file_extension": ".py",
87 | "mimetype": "text/x-python",
88 | "name": "python",
89 | "nbconvert_exporter": "python",
90 | "pygments_lexer": "ipython3",
91 | "version": "3.7.3"
92 | }
93 | },
94 | "nbformat": 4,
95 | "nbformat_minor": 2
96 | }
97 |
--------------------------------------------------------------------------------
/example_notebooks/serving/triton/sentence_classification_app/clients/deepset.py:
--------------------------------------------------------------------------------
1 | import numpy as np
2 | import sys
3 | from functools import partial
4 | import os
5 | import tritonclient.grpc as tritongrpcclient
6 | import tritonclient.grpc.model_config_pb2 as mc
7 | import tritonclient.http as tritonhttpclient
8 | from tritonclient.utils import triton_to_np_dtype
9 | from tritonclient.utils import InferenceServerException
10 | import torch
11 | from transformers import AutoTokenizer
12 | from torch.nn import functional as F
13 | import json
14 |
15 | tokenizer = AutoTokenizer.from_pretrained('deepset/sentence_bert')
16 | VERBOSE = False
17 | # sentence1 = 'Who are you voting for 2021?'
18 | # sentence2 = 'Jupiter’s Biggest Moons Started as Tiny Grains of Hail'
19 | # sentence3 = 'Hi Matt, your payment is one week past due. Please use the link below to make your payment'
20 | labels = ['business', 'space and science', 'politics']
21 | input_name = ['input__0', 'input__1']
22 | output_name = 'output__0'
23 |
24 |
25 | def run_inference(sentence, model_name='deepset', url='hyperplane-triton.default:8000', model_version='1'):
26 | triton_client = tritonhttpclient.InferenceServerClient(
27 | url=url, verbose=VERBOSE)
28 | model_metadata = triton_client.get_model_metadata(
29 | model_name=model_name, model_version=model_version)
30 | model_config = triton_client.get_model_config(
31 | model_name=model_name, model_version=model_version)
32 | # I have restricted the input sequence length to 256
33 | inputs = tokenizer.batch_encode_plus([sentence] + labels,
34 | return_tensors='pt', max_length=256,
35 | truncation=True, padding='max_length')
36 |
37 | input_ids = inputs['input_ids']
38 | input_ids = np.array(input_ids, dtype=np.int32)
39 | mask = inputs['attention_mask']
40 | mask = np.array(mask, dtype=np.int32)
41 | mask = mask.reshape(4, 256)
42 | input_ids = input_ids.reshape(4, 256)
43 | input0 = tritonhttpclient.InferInput(input_name[0], (4, 256), 'INT32')
44 | input0.set_data_from_numpy(input_ids, binary_data=False)
45 | input1 = tritonhttpclient.InferInput(input_name[1], (4, 256), 'INT32')
46 | input1.set_data_from_numpy(mask, binary_data=False)
47 | output = tritonhttpclient.InferRequestedOutput(output_name, binary_data=False)
48 | response = triton_client.infer(model_name, model_version=model_version, inputs=[input0, input1], outputs=[output])
49 | embeddings = response.as_numpy('output__0')
50 | embeddings = torch.from_numpy(embeddings)
51 | sentence_rep = embeddings[:1].mean(dim=1)
52 | label_reps = embeddings[1:].mean(dim=1)
53 | similarities = F.cosine_similarity(sentence_rep, label_reps)
54 | closest = similarities.argsort(descending=True)
55 | # results = []
56 | # for ind in closest:
57 | # results.append(f'label: {labels[ind]} \t similarity: {similarities[ind]}')
58 | # return json.dumps(results)
59 | return labels[closest[0]]
60 |
61 | if __name__ == '__main__':
62 | import sys
63 | sent = sys.argv[1]
64 | print(run_inference(sys.argv[1]))
65 |
66 |
67 | # def run(string="", url="hyperplane-triton.default:8000"):
68 | # results = run_inference(sentence=string, url=url)
69 | # return results
--------------------------------------------------------------------------------
/LLMs/confluence_app/confluence_qa.py:
--------------------------------------------------------------------------------
1 | import os
2 | from langchain.document_loaders import ConfluenceLoader
3 | from langchain.text_splitter import CharacterTextSplitter, TokenTextSplitter
4 | from langchain.embeddings.openai import OpenAIEmbeddings
5 | from langchain.prompts import PromptTemplate
6 | from langchain.chat_models import ChatOpenAI
7 | from constants import *
8 |
9 | from langchain.vectorstores import Chroma
10 | from langchain.chains import RetrievalQA
11 |
12 | class ConfluenceQA:
13 | def __init__(self,config:dict = {}):
14 | self.config = config
15 | self.embedding = None
16 | self.vectordb = None
17 | self.llm = None
18 | self.qa = None
19 | self.retriever = None
20 | def init_embeddings(self) -> None:
21 | # OpenAI ada embeddings API
22 | self.embedding = OpenAIEmbeddings()
23 | def init_models(self) -> None:
24 | # OpenAI GPT 3.5 API
25 | self.llm = ChatOpenAI(model_name=LLM_OPENAI_GPT35, temperature=0.)
26 |
27 | def vector_db_confluence_docs(self,force_reload:bool= False) -> None:
28 | """
29 | creates vector db for the embeddings and persists them or loads a vector db from the persist directory
30 | """
31 | persist_directory = self.config.get("persist_directory",None)
32 | confluence_url = self.config.get("confluence_url",None)
33 | username = self.config.get("username",None)
34 | api_key = self.config.get("api_key",None)
35 | space_key = self.config.get("space_key",None)
36 | if persist_directory and os.path.exists(persist_directory) and not force_reload:
37 | ## Load from the persist db
38 | self.vectordb = Chroma(persist_directory=persist_directory, embedding_function=self.embedding)
39 | else:
40 | ## 1. Extract the documents
41 | loader = ConfluenceLoader(
42 | url=confluence_url,
43 | username = username,
44 | api_key= api_key
45 | )
46 | documents = loader.load(
47 | space_key=space_key,
48 | limit=100)
49 | ## 2. Split the texts
50 | text_splitter = CharacterTextSplitter(chunk_size=100, chunk_overlap=0)
51 | texts = text_splitter.split_documents(documents)
52 | text_splitter = TokenTextSplitter(chunk_size=1000, chunk_overlap=10, encoding_name="cl100k_base") # This the encoding for text-embedding-ada-002
53 | texts = text_splitter.split_documents(texts)
54 |
55 | ## 3. Create Embeddings and add to chroma store
56 | ##TODO: Validate if self.embedding is not None
57 | self.vectordb = Chroma.from_documents(documents=texts, embedding=self.embedding, persist_directory=persist_directory)
58 | def retreival_qa_chain(self):
59 | """
60 | Creates retrieval qa chain using vectordb as retrivar and LLM to complete the prompt
61 | """
62 | ##TODO: Use custom prompt
63 | self.retriever = self.vectordb.as_retriever(search_kwargs={"k":4})
64 | self.qa = RetrievalQA.from_chain_type(llm=self.llm, chain_type="stuff",retriever=self.retriever)
65 |
66 | def answer_confluence(self,question:str) ->str:
67 | """
68 | Answer the question
69 | """
70 | answer = self.qa.run(question)
71 | return answer
72 |
--------------------------------------------------------------------------------
/example_notebooks/frontend/voila/multiple_notebooks/ipympl.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "metadata": {},
6 | "source": [
7 | "# So easy, *voilà*!\n",
8 | "\n",
9 | "In this example notebook, we demonstrate how Voilà can render custom interactive matplotlib figures using the [ipympl](https://github.com/matplotlib/ipympl) widget."
10 | ]
11 | },
12 | {
13 | "cell_type": "code",
14 | "execution_count": null,
15 | "metadata": {},
16 | "outputs": [],
17 | "source": [
18 | "%matplotlib widget\n",
19 | "import ipympl\n",
20 | "\n",
21 | "import numpy as np\n",
22 | "import matplotlib.pyplot as plt\n",
23 | "\n",
24 | "x = np.linspace(0, 2 * np.pi, 500)\n",
25 | "y1 = np.sin(x)\n",
26 | "y2 = np.sin(3 * x)\n",
27 | "\n",
28 | "fig, ax = plt.subplots()\n",
29 | "ax.fill(x, y1, 'b', x, y2, 'r', alpha=0.3)\n",
30 | "plt.show()"
31 | ]
32 | },
33 | {
34 | "cell_type": "code",
35 | "execution_count": null,
36 | "metadata": {},
37 | "outputs": [],
38 | "source": [
39 | "import numpy as np\n",
40 | "import matplotlib.pyplot as plt\n",
41 | "\n",
42 | "plt.style.use('ggplot')\n",
43 | "\n",
44 | "fig, axes = plt.subplots(ncols=2, nrows=2)\n",
45 | "ax1, ax2, ax3, ax4 = axes.ravel()\n",
46 | "\n",
47 | "# scatter plot (Note: `plt.scatter` doesn't use default colors)\n",
48 | "x, y = np.random.normal(size=(2, 200))\n",
49 | "ax1.plot(x, y, 'o')\n",
50 | "\n",
51 | "# sinusoidal lines with colors from default color cycle\n",
52 | "L = 2 * np.pi\n",
53 | "x = np.linspace(0, L)\n",
54 | "ncolors = len(plt.rcParams['axes.prop_cycle'])\n",
55 | "shift = np.linspace(0, L, ncolors, endpoint=False)\n",
56 | "for s in shift:\n",
57 | " ax2.plot(x, np.sin(x + s), '-')\n",
58 | "ax2.margins(0)\n",
59 | "\n",
60 | "# bar graphs\n",
61 | "x = np.arange(5)\n",
62 | "y1, y2 = np.random.randint(1, 25, size=(2, 5))\n",
63 | "width = 0.25\n",
64 | "ax3.bar(x, y1, width)\n",
65 | "ax3.bar(x + width, y2, width,\n",
66 | " color=list(plt.rcParams['axes.prop_cycle'])[2]['color'])\n",
67 | "ax3.set_xticks(x + width)\n",
68 | "ax3.set_xticklabels(['a', 'b', 'c', 'd', 'e'])\n",
69 | "\n",
70 | "# circles with colors from default color cycle\n",
71 | "for i, color in enumerate(plt.rcParams['axes.prop_cycle']):\n",
72 | " xy = np.random.normal(size=2)\n",
73 | " ax4.add_patch(plt.Circle(xy, radius=0.3, color=color['color']))\n",
74 | "\n",
75 | "ax4.axis('equal')\n",
76 | "ax4.margins(0)\n",
77 | "\n",
78 | "plt.show()"
79 | ]
80 | },
81 | {
82 | "cell_type": "code",
83 | "execution_count": null,
84 | "metadata": {},
85 | "outputs": [],
86 | "source": []
87 | }
88 | ],
89 | "metadata": {
90 | "kernelspec": {
91 | "display_name": "Python 3",
92 | "language": "python",
93 | "name": "python3"
94 | },
95 | "language_info": {
96 | "codemirror_mode": {
97 | "name": "ipython",
98 | "version": 3
99 | },
100 | "file_extension": ".py",
101 | "mimetype": "text/x-python",
102 | "name": "python",
103 | "nbconvert_exporter": "python",
104 | "pygments_lexer": "ipython3",
105 | "version": "3.7.3"
106 | }
107 | },
108 | "nbformat": 4,
109 | "nbformat_minor": 2
110 | }
111 |
--------------------------------------------------------------------------------
/example_notebooks/utils/ray_starter.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "code",
5 | "execution_count": null,
6 | "id": "31f5f060-610d-44d1-ac83-938c82870904",
7 | "metadata": {},
8 | "outputs": [],
9 | "source": [
10 | "from hyperplane.ray_common import initialize_ray_cluster, stop_ray_cluster, find_ray_workers\n",
11 | "num_workers = 2\n",
12 | "cpu_core_per_worker = 15\n",
13 | "ram_gb_per_worker = 12 #110 GB allocatible for 16_128 nodes, 12 for 16_16 nodes, 27 for 32_32 nodes\n",
14 | "ray_cluster = initialize_ray_cluster(num_workers, cpu_core_per_worker, ram_gb_per_worker)"
15 | ]
16 | },
17 | {
18 | "cell_type": "code",
19 | "execution_count": null,
20 | "id": "42fc1e86-ee3e-4b18-8b3a-2a7a73b25b61",
21 | "metadata": {},
22 | "outputs": [],
23 | "source": [
24 | "import ray\n",
25 | "# Create a Dataset of Python objects.\n",
26 | "ds = ray.data.range(10000)\n",
27 | "ds"
28 | ]
29 | },
30 | {
31 | "cell_type": "code",
32 | "execution_count": null,
33 | "id": "a8561398-4bc6-43c5-aab5-516f156c8e02",
34 | "metadata": {},
35 | "outputs": [],
36 | "source": [
37 | "import pyarrow\n",
38 | "pyarrow.__version__"
39 | ]
40 | },
41 | {
42 | "cell_type": "code",
43 | "execution_count": null,
44 | "id": "724f59e5-a70a-4ce2-8eef-e598b106af75",
45 | "metadata": {},
46 | "outputs": [],
47 | "source": [
48 | "ds.take(5)"
49 | ]
50 | },
51 | {
52 | "cell_type": "code",
53 | "execution_count": null,
54 | "id": "5386dfd4-7073-4165-8a5d-da346705a23a",
55 | "metadata": {},
56 | "outputs": [],
57 | "source": [
58 | "ds.count()"
59 | ]
60 | },
61 | {
62 | "cell_type": "code",
63 | "execution_count": null,
64 | "id": "8710a815-db9e-4bc6-a477-a5cc21ef1dbd",
65 | "metadata": {},
66 | "outputs": [],
67 | "source": [
68 | "# Create a Dataset of Arrow records.\n",
69 | "ds = ray.data.from_items([{\"col1\": i, \"col2\": str(i)} for i in range(10000)])"
70 | ]
71 | },
72 | {
73 | "cell_type": "code",
74 | "execution_count": null,
75 | "id": "db5cc7ea-12d3-4ca8-a597-47058695e71c",
76 | "metadata": {},
77 | "outputs": [],
78 | "source": [
79 | "ds.show(5)"
80 | ]
81 | },
82 | {
83 | "cell_type": "code",
84 | "execution_count": null,
85 | "id": "6b013285-e164-493e-80e0-32f7885e4cf6",
86 | "metadata": {},
87 | "outputs": [],
88 | "source": [
89 | "ds.schema()"
90 | ]
91 | },
92 | {
93 | "cell_type": "code",
94 | "execution_count": null,
95 | "id": "335814b0-e935-4f75-8a9b-a49cf3b19996",
96 | "metadata": {},
97 | "outputs": [],
98 | "source": [
99 | "stop_ray_cluster(ray_cluster)"
100 | ]
101 | },
102 | {
103 | "cell_type": "code",
104 | "execution_count": null,
105 | "id": "91312729-f717-4655-8e8e-1607564b03ac",
106 | "metadata": {},
107 | "outputs": [],
108 | "source": []
109 | }
110 | ],
111 | "metadata": {
112 | "kernelspec": {
113 | "display_name": "Python [conda env:root] *",
114 | "language": "python",
115 | "name": "conda-root-py"
116 | },
117 | "language_info": {
118 | "codemirror_mode": {
119 | "name": "ipython",
120 | "version": 3
121 | },
122 | "file_extension": ".py",
123 | "mimetype": "text/x-python",
124 | "name": "python",
125 | "nbconvert_exporter": "python",
126 | "pygments_lexer": "ipython3",
127 | "version": "3.8.10"
128 | }
129 | },
130 | "nbformat": 4,
131 | "nbformat_minor": 5
132 | }
133 |
--------------------------------------------------------------------------------
/LLMs/confluence_app/app.py:
--------------------------------------------------------------------------------
1 | import streamlit as st
2 | import os
3 | import json
4 | import time
5 | from dotenv import load_dotenv
6 |
7 | # Import the ConfluenceQA class
8 | from confluence_qa import ConfluenceQA
9 |
10 | try:
11 | from hyperplane.utils import is_jhub
12 | if is_jhub():
13 | openaiKeyFile = '/root/.secret/openai_key.json'
14 | else:
15 | openaiKeyFile = '/etc/hyperplane/secrets/openai_key.json'
16 | with open(openaiKeyFile) as f:
17 | os.environ["OPENAI_API_KEY"] = json.load(f)['openai_key']
18 | except Exception as e:
19 | print(e)
20 | load_dotenv()
21 |
22 | st.set_page_config(
23 | page_title='Q&A Bot for Confluence Page',
24 | page_icon='⚡',
25 | layout='wide',
26 | initial_sidebar_state='auto',
27 | )
28 | if "config" not in st.session_state:
29 | st.session_state["config"] = {}
30 | if "confluence_qa" not in st.session_state:
31 | st.session_state["confluence_qa"] = None
32 |
33 | @st.cache_resource
34 | def load_confluence(config):
35 | # st.write("loading the confluence page")
36 | confluence_qa = ConfluenceQA(config=config)
37 | confluence_qa.init_embeddings()
38 | confluence_qa.init_models()
39 | confluence_qa.vector_db_confluence_docs()
40 | confluence_qa.retreival_qa_chain()
41 | return confluence_qa
42 |
43 | with st.sidebar.form(key ='Form1'):
44 | st.markdown('## Add your configs')
45 | confluence_url = st.text_input("paste the confluence URL", "https://templates.atlassian.net/wiki/")
46 | username = st.text_input(label="confluence username",
47 | help="leave blank if confluence page is public",
48 | type="password")
49 | space_key = st.text_input(label="confluence space",
50 | help="Space of Confluence",
51 | value="RD")
52 | api_key = st.text_input(label="confluence api key",
53 | help="leave blank if confluence page is public",
54 | type="password")
55 | submitted1 = st.form_submit_button(label='Submit')
56 |
57 | if submitted1 and confluence_url and space_key:
58 | st.session_state["config"] = {
59 | "persist_directory": None,
60 | "confluence_url": confluence_url,
61 | "username": username if username != "" else None,
62 | "api_key": api_key if api_key != "" else None,
63 | "space_key": space_key,
64 | }
65 | with st.spinner(text="Ingesting Confluence..."):
66 | ### Hardcoding for https://templates.atlassian.net/wiki/ and space RD to avoid multiple OpenAI calls.
67 | config = st.session_state["config"]
68 | if config["confluence_url"] == "https://templates.atlassian.net/wiki/" and config["space_key"] =="RD":
69 | config["persist_directory"] = "chroma_db"
70 | st.session_state["config"] = config
71 |
72 | st.session_state["confluence_qa"] = load_confluence(st.session_state["config"])
73 | st.write("Confluence Space Ingested")
74 |
75 |
76 | st.title("Confluence Q&A Demo")
77 |
78 | question = st.text_input('Ask a question', "How do I make a space public?")
79 |
80 | if st.button('Get Answer', key='button2'):
81 | with st.spinner(text="Asking LLM..."):
82 | confluence_qa = st.session_state.get("confluence_qa")
83 | if confluence_qa is not None:
84 | result = confluence_qa.answer_confluence(question)
85 | st.write(result)
86 | else:
87 | st.write("Please load Confluence page first.")
--------------------------------------------------------------------------------
/LLMs/QA_app/streamlit_app_blog.py:
--------------------------------------------------------------------------------
1 | import streamlit as st
2 | from pdf_qa import PdfQA
3 | from pathlib import Path
4 | from tempfile import NamedTemporaryFile
5 | import time
6 | import shutil
7 | from constants import *
8 |
9 |
10 |
11 | # Streamlit app code
12 | st.set_page_config(
13 | page_title='Q&A Bot for PDF',
14 | page_icon='🔖',
15 | layout='wide',
16 | initial_sidebar_state='auto',
17 | )
18 |
19 |
20 | if "pdf_qa_model" not in st.session_state:
21 | st.session_state["pdf_qa_model"]:PdfQA = PdfQA() ## Intialisation
22 |
23 | ## To cache resource across multiple session
24 | @st.cache_resource
25 | def load_llm(llm,load_in_8bit):
26 |
27 | if llm == LLM_OPENAI_GPT35:
28 | pass
29 | elif llm == LLM_FLAN_T5_SMALL:
30 | return PdfQA.create_flan_t5_small(load_in_8bit)
31 | elif llm == LLM_FLAN_T5_BASE:
32 | return PdfQA.create_flan_t5_base(load_in_8bit)
33 | elif llm == LLM_FLAN_T5_LARGE:
34 | return PdfQA.create_flan_t5_large(load_in_8bit)
35 | elif llm == LLM_FASTCHAT_T5_XL:
36 | return PdfQA.create_fastchat_t5_xl(load_in_8bit)
37 | elif llm == LLM_FALCON_SMALL:
38 | return PdfQA.create_falcon_instruct_small(load_in_8bit)
39 | else:
40 | raise ValueError("Invalid LLM setting")
41 |
42 | ## To cache resource across multiple session
43 | @st.cache_resource
44 | def load_emb(emb):
45 | if emb == EMB_INSTRUCTOR_XL:
46 | return PdfQA.create_instructor_xl()
47 | elif emb == EMB_SBERT_MPNET_BASE:
48 | return PdfQA.create_sbert_mpnet()
49 | elif emb == EMB_SBERT_MINILM:
50 | pass ##ChromaDB takes care
51 | else:
52 | raise ValueError("Invalid embedding setting")
53 |
54 |
55 |
56 | st.title("PDF Q&A (Self hosted LLMs)")
57 |
58 | with st.sidebar:
59 | emb = st.radio("**Select Embedding Model**", [EMB_INSTRUCTOR_XL, EMB_SBERT_MPNET_BASE,EMB_SBERT_MINILM],index=1)
60 | llm = st.radio("**Select LLM Model**", [LLM_FASTCHAT_T5_XL, LLM_FLAN_T5_SMALL,LLM_FLAN_T5_BASE,LLM_FLAN_T5_LARGE,LLM_FLAN_T5_XL,LLM_FALCON_SMALL],index=2)
61 | load_in_8bit = st.radio("**Load 8 bit**", [True, False],index=1)
62 | pdf_file = st.file_uploader("**Upload PDF**", type="pdf")
63 |
64 |
65 | if st.button("Submit") and pdf_file is not None:
66 | with st.spinner(text="Uploading PDF and Generating Embeddings.."):
67 | with NamedTemporaryFile(delete=False, suffix='.pdf') as tmp:
68 | shutil.copyfileobj(pdf_file, tmp)
69 | tmp_path = Path(tmp.name)
70 | st.session_state["pdf_qa_model"].config = {
71 | "pdf_path": str(tmp_path),
72 | "embedding": emb,
73 | "llm": llm,
74 | "load_in_8bit": load_in_8bit
75 | }
76 | st.session_state["pdf_qa_model"].embedding = load_emb(emb)
77 | st.session_state["pdf_qa_model"].llm = load_llm(llm,load_in_8bit)
78 | st.session_state["pdf_qa_model"].init_embeddings()
79 | st.session_state["pdf_qa_model"].init_models()
80 | st.session_state["pdf_qa_model"].vector_db_pdf()
81 | st.sidebar.success("PDF uploaded successfully")
82 |
83 | question = st.text_input('Ask a question', 'What is this document?')
84 |
85 | if st.button("Answer"):
86 | try:
87 | st.session_state["pdf_qa_model"].retreival_qa_chain()
88 | answer = st.session_state["pdf_qa_model"].answer_query(question)
89 | st.write(f"{answer}")
90 | except Exception as e:
91 | st.error(f"Error answering the question: {str(e)}")
--------------------------------------------------------------------------------
/example_notebooks/utils/dask_snowflake.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "id": "248d39d3",
6 | "metadata": {},
7 | "source": [
8 | "## A boilerplate code for using Dask to read from Snowflake "
9 | ]
10 | },
11 | {
12 | "cell_type": "code",
13 | "execution_count": null,
14 | "id": "0657a24f",
15 | "metadata": {},
16 | "outputs": [],
17 | "source": [
18 | "import dask.dataframe as dd\n",
19 | "df = dd.read_sql_table(\n",
20 | " 'accounts', \n",
21 | " 'snowflake://user:pass@...warehouse=...role=...', \n",
22 | " npartitions=10, \n",
23 | " index_col='id'\n",
24 | ")"
25 | ]
26 | },
27 | {
28 | "cell_type": "code",
29 | "execution_count": null,
30 | "id": "063437a0-0743-424c-867f-9f9fed346e3a",
31 | "metadata": {},
32 | "outputs": [],
33 | "source": [
34 | "## Please checkout the API documentation https://yourdomain.hyperplane.dev/hyperplane_docs/api.html for more details\n",
35 | "\n",
36 | "from hyperplane import notebook_common as nc\n",
37 | "num_workers = 2 # number of nodes to spin up\n",
38 | "\n",
39 | "## node specific parameters\n",
40 | "total_memory = 110 #110 GB allocatible for 16_128 nodes, 12G for 16_16 nodes, 27G for 32_32\n",
41 | "cors_per_worker = 15 # 15 cores for 16_128 nodes and 16_16 nodes, 28 cores for 32_32 nodes\n",
42 | "nprocs = 15\n",
43 | "ram_gb_per_proc = total_memory/nprocs\n",
44 | "nthreads = int(cors_per_worker/nprocs)\n",
45 | "\n",
46 | "print(f'initializing with {num_workers} num_workers, {nprocs} nprocs each proc has {ram_gb_per_proc} GB')\n",
47 | "client, cluster = nc.initialize_cluster(\n",
48 | " num_workers = num_workers,\n",
49 | " nprocs = nprocs,\n",
50 | " nthreads = nthreads,\n",
51 | " ram_gb_per_proc = ram_gb_per_proc,\n",
52 | " cores_per_worker = cors_per_worker\n",
53 | " )"
54 | ]
55 | },
56 | {
57 | "cell_type": "code",
58 | "execution_count": null,
59 | "id": "490df05b",
60 | "metadata": {},
61 | "outputs": [],
62 | "source": [
63 | "import snowflake.connector\n",
64 | "from dask.dataframe import from_delayed\n",
65 | "from dask.distributed import delayed\n",
66 | "\n",
67 | "@delayed\n",
68 | "def load(connection_info, query, start, end):\n",
69 | " conn = snowflake.connector.connect(**connection_info)\n",
70 | " cur = conn.cursor()\n",
71 | " cur.execute(query, start, end)\n",
72 | " return cur.fetch_pandas_all()\n",
73 | "ddf = from_delayed(*[load(connection_info, query, st, ed) for st, ed in partitions])\n",
74 | "ddf.persist()"
75 | ]
76 | },
77 | {
78 | "cell_type": "code",
79 | "execution_count": null,
80 | "id": "07b1394c-0157-4148-a978-a1a0a64a1047",
81 | "metadata": {},
82 | "outputs": [],
83 | "source": [
84 | "client.close()\n",
85 | "cluster.close()"
86 | ]
87 | }
88 | ],
89 | "metadata": {
90 | "kernelspec": {
91 | "display_name": "Python 3 (ipykernel)",
92 | "language": "python",
93 | "name": "python3"
94 | },
95 | "language_info": {
96 | "codemirror_mode": {
97 | "name": "ipython",
98 | "version": 3
99 | },
100 | "file_extension": ".py",
101 | "mimetype": "text/x-python",
102 | "name": "python",
103 | "nbconvert_exporter": "python",
104 | "pygments_lexer": "ipython3",
105 | "version": "3.8.10"
106 | }
107 | },
108 | "nbformat": 4,
109 | "nbformat_minor": 5
110 | }
111 |
--------------------------------------------------------------------------------
/example_notebooks/frontend/voila/multiple_notebooks/dashboard.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "metadata": {},
6 | "source": [
7 | "This demo uses Voilà to render a notebook to a custom HTML page using gridstack.js for the layout of each output. In the cell metadata you can change the default cell with and height (in grid units between 1 and 12) by specifying.\n",
8 | " * `grid_row`\n",
9 | " * `grid_columns`"
10 | ]
11 | },
12 | {
13 | "cell_type": "code",
14 | "execution_count": null,
15 | "metadata": {},
16 | "outputs": [],
17 | "source": [
18 | "import numpy as np\n",
19 | "n = 200\n",
20 | "\n",
21 | "x = np.linspace(0.0, 10.0, n)\n",
22 | "y = np.cumsum(np.random.randn(n)*10).astype(int)\n"
23 | ]
24 | },
25 | {
26 | "cell_type": "code",
27 | "execution_count": null,
28 | "metadata": {},
29 | "outputs": [],
30 | "source": [
31 | "import ipywidgets as widgets"
32 | ]
33 | },
34 | {
35 | "cell_type": "code",
36 | "execution_count": null,
37 | "metadata": {},
38 | "outputs": [],
39 | "source": [
40 | "label_selected = widgets.Label(value=\"Selected: 0\")\n",
41 | "label_selected"
42 | ]
43 | },
44 | {
45 | "cell_type": "code",
46 | "execution_count": null,
47 | "metadata": {
48 | "grid_columns": 8,
49 | "grid_rows": 4
50 | },
51 | "outputs": [],
52 | "source": [
53 | "import numpy as np\n",
54 | "from bqplot import pyplot as plt\n",
55 | "import bqplot\n",
56 | "\n",
57 | "fig = plt.figure( title='Histogram')\n",
58 | "np.random.seed(0)\n",
59 | "hist = plt.hist(y, bins=25)\n",
60 | "hist.scales['sample'].min = float(y.min())\n",
61 | "hist.scales['sample'].max = float(y.max())\n",
62 | "display(fig)\n",
63 | "fig.layout.width = 'auto'\n",
64 | "fig.layout.height = 'auto'\n",
65 | "fig.layout.min_height = '300px' # so it shows nicely in the notebook\n",
66 | "fig.layout.flex = '1'"
67 | ]
68 | },
69 | {
70 | "cell_type": "code",
71 | "execution_count": null,
72 | "metadata": {
73 | "grid_columns": 12,
74 | "grid_rows": 6
75 | },
76 | "outputs": [],
77 | "source": [
78 | "import numpy as np\n",
79 | "from bqplot import pyplot as plt\n",
80 | "import bqplot\n",
81 | "\n",
82 | "fig = plt.figure( title='Line Chart')\n",
83 | "np.random.seed(0)\n",
84 | "n = 200\n",
85 | "p = plt.plot(x, y)\n",
86 | "fig"
87 | ]
88 | },
89 | {
90 | "cell_type": "code",
91 | "execution_count": null,
92 | "metadata": {},
93 | "outputs": [],
94 | "source": [
95 | "fig.layout.width = 'auto'\n",
96 | "fig.layout.height = 'auto'\n",
97 | "fig.layout.min_height = '300px' # so it shows nicely in the notebook\n",
98 | "fig.layout.flex = '1'"
99 | ]
100 | },
101 | {
102 | "cell_type": "code",
103 | "execution_count": null,
104 | "metadata": {},
105 | "outputs": [],
106 | "source": [
107 | "brushintsel = bqplot.interacts.BrushIntervalSelector(scale=p.scales['x'])"
108 | ]
109 | },
110 | {
111 | "cell_type": "code",
112 | "execution_count": null,
113 | "metadata": {},
114 | "outputs": [],
115 | "source": [
116 | "def update_range(*args):\n",
117 | " label_selected.value = \"Selected range {}\".format(brushintsel.selected)\n",
118 | " mask = (x > brushintsel.selected[0]) & (x < brushintsel.selected[1])\n",
119 | " hist.sample = y[mask]\n",
120 | " \n",
121 | "brushintsel.observe(update_range, 'selected')\n",
122 | "fig.interaction = brushintsel"
123 | ]
124 | }
125 | ],
126 | "metadata": {
127 | "celltoolbar": "Edit Metadata",
128 | "kernelspec": {
129 | "display_name": "Python 3",
130 | "language": "python",
131 | "name": "python3"
132 | },
133 | "language_info": {
134 | "codemirror_mode": {
135 | "name": "ipython",
136 | "version": 3
137 | },
138 | "file_extension": ".py",
139 | "mimetype": "text/x-python",
140 | "name": "python",
141 | "nbconvert_exporter": "python",
142 | "pygments_lexer": "ipython3",
143 | "version": "3.6.4"
144 | }
145 | },
146 | "nbformat": 4,
147 | "nbformat_minor": 2
148 | }
149 |
--------------------------------------------------------------------------------
/example_notebooks/utils/submit_graphql_with_python.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "id": "66400b02",
6 | "metadata": {},
7 | "source": [
8 | "\n",
9 | "### Example of submit GraphQL query with the `graphql_operation` function"
10 | ]
11 | },
12 | {
13 | "cell_type": "code",
14 | "execution_count": 7,
15 | "id": "7a025c40",
16 | "metadata": {},
17 | "outputs": [],
18 | "source": [
19 | "from hyperplane.notebook_common import graphql_operation\n",
20 | "import numpy as np"
21 | ]
22 | },
23 | {
24 | "cell_type": "markdown",
25 | "id": "237e53e3",
26 | "metadata": {},
27 | "source": [
28 | "## submit one job"
29 | ]
30 | },
31 | {
32 | "cell_type": "code",
33 | "execution_count": 36,
34 | "id": "ba283457",
35 | "metadata": {},
36 | "outputs": [],
37 | "source": [
38 | "gql_query = \"\"\"\n",
39 | " mutation submitJob {\n",
40 | " createPipelineJob(\n",
41 | " data: {\n",
42 | " jobType: \"basic\"\n",
43 | " timeout: 1800\n",
44 | " activeTimeout: 1800\n",
45 | " maxRetries: 1\n",
46 | " pipelineYamlPath: \"crypto-bot/balance_pipeline/get_balances.yaml\"\n",
47 | " }\n",
48 | " ) {\n",
49 | " id\n",
50 | " status\n",
51 | " }\n",
52 | " }\n",
53 | "\n",
54 | "\"\"\" "
55 | ]
56 | },
57 | {
58 | "cell_type": "code",
59 | "execution_count": 37,
60 | "id": "19d0a352",
61 | "metadata": {},
62 | "outputs": [
63 | {
64 | "name": "stdout",
65 | "output_type": "stream",
66 | "text": [
67 | "[{'createPipelineJob': {'id': 'cdd1e16c-9e42-4d2d-add7-12d974cf0d08', 'status': 'pending'}}]\n"
68 | ]
69 | }
70 | ],
71 | "source": [
72 | "### the graphql_operation funcition is a async function so we add await. \n",
73 | "data = await graphql_operation(gql_query)\n",
74 | "print(data)"
75 | ]
76 | },
77 | {
78 | "cell_type": "markdown",
79 | "id": "1b5098b2",
80 | "metadata": {},
81 | "source": [
82 | "## submit with parameters"
83 | ]
84 | },
85 | {
86 | "cell_type": "code",
87 | "execution_count": 32,
88 | "id": "76063976",
89 | "metadata": {},
90 | "outputs": [
91 | {
92 | "data": {
93 | "text/plain": [
94 | "[{'createPipelineJob': {'id': 'bf63b1c4-1a92-4ce3-bbac-e2e590241c3e',\n",
95 | " 'status': 'pending'}}]"
96 | ]
97 | },
98 | "execution_count": 32,
99 | "metadata": {},
100 | "output_type": "execute_result"
101 | }
102 | ],
103 | "source": [
104 | "timeout = 180\n",
105 | "max_retries = 1\n",
106 | "param = \"param1\"\n",
107 | "\n",
108 | "gql_query = \"\"\"\n",
109 | " mutation submitModel { \n",
110 | " createPipelineJob (data: {\n",
111 | " jobType: \"basic\",\n",
112 | " timeout: %d,\n",
113 | " maxRetries: %d,\n",
114 | " pipelineYamlPath: \"example.yaml\",\n",
115 | " parameters: {\n",
116 | " create: [\n",
117 | " {key: \"param1\", value: \"%s\"},\n",
118 | " ]\n",
119 | " }\n",
120 | " }) { \n",
121 | " id \n",
122 | " status\n",
123 | "\n",
124 | " } \n",
125 | " }\n",
126 | " \"\"\" % (timeout, max_retries, param)\n",
127 | "\n",
128 | "data = await graphql_operation(gql_query)\n",
129 | "data"
130 | ]
131 | },
132 | {
133 | "cell_type": "code",
134 | "execution_count": null,
135 | "id": "00097b87",
136 | "metadata": {},
137 | "outputs": [],
138 | "source": []
139 | }
140 | ],
141 | "metadata": {
142 | "kernelspec": {
143 | "display_name": "Python [conda env:root] *",
144 | "language": "python",
145 | "name": "conda-root-py"
146 | },
147 | "language_info": {
148 | "codemirror_mode": {
149 | "name": "ipython",
150 | "version": 3
151 | },
152 | "file_extension": ".py",
153 | "mimetype": "text/x-python",
154 | "name": "python",
155 | "nbconvert_exporter": "python",
156 | "pygments_lexer": "ipython3",
157 | "version": "3.8.10"
158 | }
159 | },
160 | "nbformat": 4,
161 | "nbformat_minor": 5
162 | }
163 |
--------------------------------------------------------------------------------
/LLMs/milvus_chatbot/service/app.py:
--------------------------------------------------------------------------------
1 | import streamlit as st
2 |
3 | import json
4 |
5 | import numpy
6 | import copy
7 |
8 | from pymilvus import Collection, utility
9 |
10 | import os
11 | from dotenv import load_dotenv
12 | load_dotenv(override=True)
13 |
14 | import langchain
15 | from langchain.embeddings import HuggingFaceEmbeddings
16 |
17 | import text_generation
18 | from langchain import PromptTemplate
19 |
20 | from pymilvus import connections
21 | connection = connections.connect(
22 | alias="default",
23 | host="milvus.hyperplane-milvus.svc.cluster.local",
24 | port=19530,
25 | )
26 |
27 | SVC_EP=os.environ['HYPERPLANE_JOB_PARAMETER_LLM_ENDPOINT']
28 | client = text_generation.Client(SVC_EP)
29 |
30 | embeddings = HuggingFaceEmbeddings(model_name='all-MiniLM-L6-v2')
31 |
32 | whcollection = Collection("WikiHow")
33 | whcollection.load()
34 |
35 | assistant_string = "ASSISTANT:\n"
36 | user_string = "USER:\n"
37 | document_string="DOCUMENT:\n"
38 |
39 | prompt_template = PromptTemplate.from_template(
40 | f"""\
41 | {{turns}}
42 | {document_string}{{context}}
43 | {user_string}What does the document say about {{prompt}}
44 | Give me a summary. If the information is not there let me know.
45 |
46 | {assistant_string}
47 | """
48 | )
49 |
50 | def generate(what, turns, context, topic, whole_context):
51 | found = whcollection.search(
52 | [embeddings.embed_query(what)],
53 | anns_field="vector",
54 | param={'metric_type': 'L2',
55 | 'offset': 0,
56 | 'params': {'nprobe': 1}
57 | },
58 | limit=1,
59 | output_fields=['text', 'title'])
60 | match_title = found[0][0].entity.get('title')
61 | match_text = found[0][0].entity.get('text')
62 | match_dist = found[0][0].distance
63 |
64 | retrieved = ""
65 |
66 | if match_title != topic and match_dist < 0.75:
67 | retrieved = match_text
68 | retrieved = retrieved[:1024]
69 | context = retrieved
70 | whole_context = match_text
71 | topic = match_title
72 | turntxt = "\n".join(turns)[-2048:]
73 | preface = ("No information available" if context is None else context)
74 |
75 | return {
76 | 'stream': client.generate_stream(prompt=prompt_template.format(turns=turntxt, prompt=what, context=preface),
77 | max_new_tokens=512,
78 | repetition_penalty=1.2,
79 | ),
80 |
81 | 'topic': topic,
82 | 'context': context,
83 | 'whole_context': whole_context
84 | }
85 |
86 | if "messages" not in st.session_state.keys():
87 | st.session_state.messages = []
88 |
89 | if "cache" not in st.session_state.keys():
90 | st.session_state.cache = [[], "", None, ""]
91 |
92 | TURNS = 0
93 | CTX = 1
94 | TOPIC = 2
95 | WHOLE_CTX = 3
96 |
97 | turns = st.session_state.cache[TURNS]
98 | context = st.session_state.cache[CTX]
99 | topic = st.session_state.cache[TOPIC]
100 | whole_context = st.session_state.cache[WHOLE_CTX]
101 |
102 | for message in st.session_state.messages:
103 | with st.chat_message(message["role"]):
104 | st.write(message["content"])
105 |
106 | ipt = st.chat_input(key="chat_query", placeholder="How can I run faster?")
107 |
108 | if ipt is not None:
109 | st.session_state.messages.append({'role': 'user', 'content': ipt})
110 | with st.chat_message("user"):
111 | st.write(ipt)
112 |
113 | if len(st.session_state.messages) != 0 and st.session_state.messages[-1]["role"] != "response":
114 | with st.chat_message("response"):
115 | with st.spinner("Thinking..."):
116 | response = generate(ipt, turns, context, topic, whole_context)
117 | st.session_state.cache[CTX] = response['context']
118 | st.session_state.cache[TOPIC] = response['topic']
119 | st.session_state.cache[WHOLE_CTX] = response['whole_context']
120 | with st.expander("Active Document"):
121 | st.write(response['whole_context'])
122 |
123 | turns.append(user_string + ipt)
124 |
125 | resp = ""
126 | placeholder = st.empty()
127 | for tok in response['stream']:
128 | if not tok.token.special:
129 | resp += tok.token.text
130 | placeholder.markdown(resp)
131 | placeholder.markdown(resp)
132 |
133 | st.session_state.cache[TURNS].append(assistant_string + resp)
134 |
135 | message = {"role": "response", "content": resp}
136 | st.session_state.messages.append(message)
137 |
--------------------------------------------------------------------------------
/example_notebooks/serving/triton/convert_pytorch.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "id": "3ef528ba",
6 | "metadata": {},
7 | "source": [
8 | "## How to trace/convert Transformer model into Triton acceptable models?\n",
9 | "- source https://github.com/sachinsharma9780/AI-Enterprise-Workshop-Building-ML-Pipelines/blob/main/"
10 | ]
11 | },
12 | {
13 | "cell_type": "code",
14 | "execution_count": null,
15 | "id": "3d57e014",
16 | "metadata": {},
17 | "outputs": [],
18 | "source": [
19 | "!pip3 install torch torchvision torchaudio --quiet"
20 | ]
21 | },
22 | {
23 | "cell_type": "code",
24 | "execution_count": null,
25 | "id": "c24f512d",
26 | "metadata": {},
27 | "outputs": [],
28 | "source": [
29 | "import torch\n",
30 | "from torch.nn import functional as F"
31 | ]
32 | },
33 | {
34 | "cell_type": "code",
35 | "execution_count": 2,
36 | "id": "e28cbf33",
37 | "metadata": {},
38 | "outputs": [
39 | {
40 | "name": "stderr",
41 | "output_type": "stream",
42 | "text": [
43 | "Some weights of the model checkpoint at distilbert-base-uncased were not used when initializing DistilBertForSequenceClassification: ['vocab_transform.weight', 'vocab_transform.bias', 'vocab_layer_norm.weight', 'vocab_layer_norm.bias', 'vocab_projector.weight', 'vocab_projector.bias']\n",
44 | "- This IS expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
45 | "- This IS NOT expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
46 | "Some weights of DistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-uncased and are newly initialized: ['pre_classifier.weight', 'pre_classifier.bias', 'classifier.weight', 'classifier.bias']\n",
47 | "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
48 | ]
49 | }
50 | ],
51 | "source": [
52 | "from transformers import DistilBertTokenizer, DistilBertForSequenceClassification\n",
53 | "tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased')\n",
54 | "model = DistilBertForSequenceClassification.from_pretrained('distilbert-base-uncased')\n",
55 | "\n",
56 | "sentence = \"Hello, my dog is cute!\"\n",
57 | "labels = \"happy\"\n"
58 | ]
59 | },
60 | {
61 | "cell_type": "code",
62 | "execution_count": null,
63 | "id": "24ec7710",
64 | "metadata": {},
65 | "outputs": [],
66 | "source": [
67 | "input_ids = tokenizer.encode(sentence, hypothesis, return_tensors = 'pt', max_length= 256, truncation = True, \n",
68 | " padding = \"max_length\")\n",
69 | "\n",
70 | "mask = input_ids !=1\n",
71 | "mask = mask.long()"
72 | ]
73 | },
74 | {
75 | "cell_type": "markdown",
76 | "id": "43d84a99",
77 | "metadata": {},
78 | "source": [
79 | "## Tracing PyTorch Model with torchscript\n",
80 | "- this is to create serializable and optimizable models, in equivalent to convert a tensorflow model with TensorRT"
81 | ]
82 | },
83 | {
84 | "cell_type": "code",
85 | "execution_count": 23,
86 | "id": "b7ec709f",
87 | "metadata": {},
88 | "outputs": [],
89 | "source": [
90 | "class PyTorch_to_TorchScript(torch.nn.Module):\n",
91 | " def __init__(self):\n",
92 | " super(PyTorch_to_TorchScript, self).__init__()\n",
93 | " self.model = AutoModelForSequenceClassification.from_pretrained('joeddav/xlm-roberta-large-xnli')\n",
94 | " def forward(self, data, attention_mask=None):\n",
95 | " return self.model(data, attention_mask)[0]"
96 | ]
97 | },
98 | {
99 | "cell_type": "code",
100 | "execution_count": 24,
101 | "id": "bd39c0aa",
102 | "metadata": {},
103 | "outputs": [],
104 | "source": [
105 | "# after trace it will save the model in cwd\n",
106 | "pt_model = PyTorch_to_TorchScript().eval()\n",
107 | "\n",
108 | "traced_script_module = torch.jit.trace(pt_model, (input_ids, attention_mask), strict=False)\n",
109 | "traced_script_module.save(\"./model.pt\")"
110 | ]
111 | },
112 | {
113 | "cell_type": "markdown",
114 | "id": "4fddef14",
115 | "metadata": {},
116 | "source": [
117 | "## Save the converted model to the Triton model repository folder"
118 | ]
119 | },
120 | {
121 | "cell_type": "code",
122 | "execution_count": 26,
123 | "id": "4610ee46",
124 | "metadata": {},
125 | "outputs": [
126 | {
127 | "data": {
128 | "text/plain": [
129 | "'/root/model_repository/sentence_bert/1/model.pt'"
130 | ]
131 | },
132 | "execution_count": 26,
133 | "metadata": {},
134 | "output_type": "execute_result"
135 | }
136 | ],
137 | "source": [
138 | "import shutil\n",
139 | "import os\n",
140 | "folder_name = 'sentence_bert'\n",
141 | "# os.mkdir(f'/root/model_repository/{folder_name}')\n",
142 | "# os.mkdir(f'/root/model_repository/{folder_name}/1')\n",
143 | "shutil.copy('model.pt', f'/root/model_repository/{folder_name}/1')"
144 | ]
145 | },
146 | {
147 | "cell_type": "code",
148 | "execution_count": null,
149 | "id": "6abe8baf",
150 | "metadata": {},
151 | "outputs": [],
152 | "source": []
153 | }
154 | ],
155 | "metadata": {
156 | "kernelspec": {
157 | "display_name": "Python [conda env:root] *",
158 | "language": "python",
159 | "name": "conda-root-py"
160 | },
161 | "language_info": {
162 | "codemirror_mode": {
163 | "name": "ipython",
164 | "version": 3
165 | },
166 | "file_extension": ".py",
167 | "mimetype": "text/x-python",
168 | "name": "python",
169 | "nbconvert_exporter": "python",
170 | "pygments_lexer": "ipython3",
171 | "version": "3.8.10"
172 | }
173 | },
174 | "nbformat": 4,
175 | "nbformat_minor": 5
176 | }
177 |
--------------------------------------------------------------------------------
/example_notebooks/utils/s3plugin.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "id": "6e82ddaa",
6 | "metadata": {},
7 | "source": [
8 | "## S3 plugin to download data to dask workers"
9 | ]
10 | },
11 | {
12 | "cell_type": "code",
13 | "execution_count": null,
14 | "id": "4258997f",
15 | "metadata": {},
16 | "outputs": [],
17 | "source": [
18 | "!pip install boto3 --quiet\n",
19 | "!pip install s3urls --quiet"
20 | ]
21 | },
22 | {
23 | "cell_type": "markdown",
24 | "id": "fb4535f4",
25 | "metadata": {},
26 | "source": [
27 | "### customized dask worker plugin for files and gz files"
28 | ]
29 | },
30 | {
31 | "cell_type": "code",
32 | "execution_count": 1,
33 | "id": "a5878f47",
34 | "metadata": {},
35 | "outputs": [],
36 | "source": [
37 | "from distributed.diagnostics.plugin import WorkerPlugin\n",
38 | "class S3DownloadPlugin(WorkerPlugin):\n",
39 | " def __init__(self, s3Url, filename):\n",
40 | " self.s3Url = s3Url\n",
41 | " self.filename = filename\n",
42 | " \n",
43 | " def setup(self, worker):\n",
44 | " self.worker = worker\n",
45 | " import boto3\n",
46 | " import click\n",
47 | " from s3urls import parse_url\n",
48 | " import tarfile\n",
49 | "\n",
50 | " import logging, traceback, sys, os \n",
51 | " logger = logging.getLogger(\"embeddings microservice\")\n",
52 | " logger.debug(\"downloading file...\")\n",
53 | " vocab_parsed_url = parse_url(self.s3Url)\n",
54 | " s3 = boto3.client('s3')\n",
55 | " s3.download_file(vocab_parsed_url['bucket'], vocab_parsed_url['key'], self.filename)\n",
56 | " logger.debug(\"done downloading....\")\n",
57 | " logger.debug(\"extracting....\")\n",
58 | " if self.filename.endswith(\"tar.gz\") or self.filename.endswith(\"tgz\"):\n",
59 | " tar = tarfile.open(self.filename, \"r:gz\")\n",
60 | " tar.extractall(path='/tmp/')\n",
61 | " tar.close()\n",
62 | " os.remove(self.filename)\n",
63 | " logger.debug(\"done extracting....\")"
64 | ]
65 | },
66 | {
67 | "cell_type": "code",
68 | "execution_count": 2,
69 | "id": "43fd1edb",
70 | "metadata": {},
71 | "outputs": [
72 | {
73 | "name": "stdout",
74 | "output_type": "stream",
75 | "text": [
76 | "s3://d2v-tmp/demo/data/aclImdb_v1.tar.gz\n"
77 | ]
78 | }
79 | ],
80 | "source": [
81 | "## this is the file you want to download to dask workers, here assumes it's sitting in a s3 bucket \n",
82 | "tgz_file_global = \"aclImdb_v1.tar.gz\"\n",
83 | "s3Url=f\"s3://d2v-tmp/demo/data/{tgz_file_global}\"\n",
84 | "filename=tgz_file_global\n",
85 | "print(s3Url)"
86 | ]
87 | },
88 | {
89 | "cell_type": "markdown",
90 | "id": "55bb1331",
91 | "metadata": {},
92 | "source": [
93 | "## Download data to local "
94 | ]
95 | },
96 | {
97 | "cell_type": "code",
98 | "execution_count": 9,
99 | "id": "ec46cc7d",
100 | "metadata": {},
101 | "outputs": [],
102 | "source": [
103 | "import boto3\n",
104 | "import click\n",
105 | "from s3urls import parse_url\n",
106 | "import tarfile\n",
107 | "\n",
108 | "import logging, traceback, sys, os \n",
109 | "vocab_parsed_url = parse_url(s3Url)\n",
110 | "s3 = boto3.client('s3')\n",
111 | "s3.download_file(vocab_parsed_url['bucket'], vocab_parsed_url['key'], filename)"
112 | ]
113 | },
114 | {
115 | "cell_type": "markdown",
116 | "id": "fee1a5e7",
117 | "metadata": {},
118 | "source": [
119 | "## Upload data to dask wokers"
120 | ]
121 | },
122 | {
123 | "cell_type": "code",
124 | "execution_count": 4,
125 | "id": "3b275fdc",
126 | "metadata": {},
127 | "outputs": [
128 | {
129 | "name": "stdout",
130 | "output_type": "stream",
131 | "text": [
132 | "👉 Hyperplane: selecting worker node pool\n",
133 | "👉 Hyperplane: selecting scheduler node pool\n",
134 | "Creating scheduler pod on cluster. This may take some time.\n",
135 | "👉 Hyperplane: spinning up a dask cluster with a scheduler as a standalone container.\n",
136 | "👉 Hyperplane: In a few minutes you'll be able to access the dashboard at https://ds.hyperplane.dev/dask-cluster-45548bff-fd74-4136-8efa-8fde70d27961/status\n",
137 | "👉 Hyperplane: to get logs from all workers, do `cluster.get_logs()`\n"
138 | ]
139 | }
140 | ],
141 | "source": [
142 | "from hyperplane import notebook_common as nc\n",
143 | "client, cluster = nc.initialize_cluster(num_workers=1)"
144 | ]
145 | },
146 | {
147 | "cell_type": "markdown",
148 | "id": "85731ca9",
149 | "metadata": {},
150 | "source": [
151 | "## Upload file to remote workers and extract if it's a compressed file\n",
152 | "- first install necessary packages on the workers \n",
153 | "- wait for 30 seconds for all workers to finish installation before moving to the next step"
154 | ]
155 | },
156 | {
157 | "cell_type": "code",
158 | "execution_count": null,
159 | "id": "df1fa98c",
160 | "metadata": {
161 | "tags": []
162 | },
163 | "outputs": [],
164 | "source": [
165 | "from dask.distributed import PipInstall\n",
166 | "import time\n",
167 | "plugin = PipInstall(packages=[\"boto3\", \"s3urls\", \"botocore\", \"click\"], pip_options=[\"--upgrade\"])\n",
168 | "client.register_worker_plugin(plugin)"
169 | ]
170 | },
171 | {
172 | "cell_type": "code",
173 | "execution_count": null,
174 | "id": "740742b1",
175 | "metadata": {},
176 | "outputs": [],
177 | "source": [
178 | "client.register_worker_plugin(S3DownloadPlugin(s3Url=s3Url, filename=tgz_file_global))"
179 | ]
180 | },
181 | {
182 | "cell_type": "code",
183 | "execution_count": 17,
184 | "id": "5616d2a8",
185 | "metadata": {},
186 | "outputs": [],
187 | "source": [
188 | "cluster.close()"
189 | ]
190 | },
191 | {
192 | "cell_type": "code",
193 | "execution_count": null,
194 | "id": "76d6cfae",
195 | "metadata": {},
196 | "outputs": [],
197 | "source": []
198 | }
199 | ],
200 | "metadata": {
201 | "kernelspec": {
202 | "display_name": "Python [conda env:root] *",
203 | "language": "python",
204 | "name": "conda-root-py"
205 | },
206 | "language_info": {
207 | "codemirror_mode": {
208 | "name": "ipython",
209 | "version": 3
210 | },
211 | "file_extension": ".py",
212 | "mimetype": "text/x-python",
213 | "name": "python",
214 | "nbconvert_exporter": "python",
215 | "pygments_lexer": "ipython3",
216 | "version": "3.8.10"
217 | }
218 | },
219 | "nbformat": 4,
220 | "nbformat_minor": 5
221 | }
222 |
--------------------------------------------------------------------------------
/example_notebooks/pipelines/crypto/watch_ob_multi.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "id": "deff5e3c-7789-4a7e-84ad-3e6da2279aaf",
6 | "metadata": {},
7 | "source": [
8 | "## Watch OrderBook of multiple ticker pair and multiple exchange asynchronous \n",
9 | "- this code runs on the `crypto` image \n",
10 | "- you can change the `slack_channel` to a your existing slack channel to get alert to the job\n",
11 | "- you can use the `pipeline.yaml` to spin up a `Hyperplane service` to watch the orderbook indefinitly "
12 | ]
13 | },
14 | {
15 | "cell_type": "code",
16 | "execution_count": 2,
17 | "id": "26a17fa6-9e70-47c7-a9fc-71528cc7be54",
18 | "metadata": {},
19 | "outputs": [],
20 | "source": [
21 | "import warnings\n",
22 | "warnings.filterwarnings(\"ignore\")\n",
23 | "\n",
24 | "import json\n",
25 | "import os\n",
26 | "import ccxtpro\n",
27 | "import ccxt\n",
28 | "import asyncio\n",
29 | "import pandas as pd\n",
30 | "import numpy as np\n",
31 | "import time\n",
32 | "from datetime import datetime, timedelta\n",
33 | "import traceback\n",
34 | "\n",
35 | "import nest_asyncio\n",
36 | "nest_asyncio.apply()\n",
37 | "\n",
38 | "## initialize a slack notification for trades or other alerts\n",
39 | "from hyperplane import notebook_common as nc\n",
40 | "sh = nc.SlackHelper()"
41 | ]
42 | },
43 | {
44 | "cell_type": "code",
45 | "execution_count": 4,
46 | "id": "afab5fb3-1676-47d5-accd-be40017e3989",
47 | "metadata": {},
48 | "outputs": [],
49 | "source": []
50 | },
51 | {
52 | "cell_type": "code",
53 | "execution_count": 5,
54 | "id": "1ef92959-ebd5-4c2d-bd1e-713c44e6d514",
55 | "metadata": {
56 | "tags": [
57 | "parameters"
58 | ]
59 | },
60 | "outputs": [],
61 | "source": [
62 | "## parameters can be injected when spijn up the jobs\n",
63 | "\n",
64 | "slack_channel = '#orderbook'\n",
65 | "\n",
66 | "exchanges = {\n",
67 | " 'kucoin': ['BTC/USDT', 'ETH/BTC', 'ETH/USDT'],\n",
68 | " 'binance': ['BTC/USDT', 'ETH/USDT','XRP/USDT']\n",
69 | " }"
70 | ]
71 | },
72 | {
73 | "cell_type": "markdown",
74 | "id": "030c37f4-5c00-4bc8-84c1-9924e6de6c37",
75 | "metadata": {},
76 | "source": [
77 | "## Watch multiple orderbooks \n",
78 | "- available apis : watch_trades, watch_ticker, watch_orderbook, watch_market, watchOHLCV"
79 | ]
80 | },
81 | {
82 | "cell_type": "code",
83 | "execution_count": 22,
84 | "id": "28052eb3-96f5-4784-9ec7-4cc6d3ddce9c",
85 | "metadata": {},
86 | "outputs": [],
87 | "source": [
88 | "orderbooks = {}\n",
89 | "\n",
90 | "def handle_all_orderbooks(orderbooks):\n",
91 | " \"\"\"your function to use the OrderBooks\"\"\"\n",
92 | "# print('We have the following orderbooks:')\n",
93 | " for exchange_id, orderbooks_by_symbol in orderbooks.items():\n",
94 | " for symbol in orderbooks_by_symbol.keys():\n",
95 | " orderbook = orderbooks_by_symbol[symbol]\n",
96 | " print(ccxtpro.Exchange.iso8601(orderbook['timestamp']), exchange_id, symbol, \n",
97 | " orderbook['asks'][0], orderbook['bids'][0])\n",
98 | " # print(orderbook)"
99 | ]
100 | },
101 | {
102 | "cell_type": "code",
103 | "execution_count": 23,
104 | "id": "6a007133-1f83-465c-bb5b-a8131d946990",
105 | "metadata": {},
106 | "outputs": [],
107 | "source": [
108 | "async def loop(asyncio_loop, exchange_id, symbol):\n",
109 | " exchange = getattr(ccxtpro, exchange_id)({\n",
110 | " 'enableRateLimit': True,\n",
111 | " 'asyncio_loop': asyncio_loop,\n",
112 | " })\n",
113 | " \n",
114 | " output_parent_path = f\"gs://{os.environ['HYPERPLANE_GCP_BUCKET']}/data/crypto_trading/{exchange_id}\"\n",
115 | " logs = []\n",
116 | " \n",
117 | " starttime = time.time()\n",
118 | " starttime_h = datetime.fromtimestamp(starttime).strftime(\"%Y-%m-%d %I:%M:%S\")\n",
119 | " sh.post_message(json.dumps({'starttime_h': \"test\"}), channel=slack_channel)\n",
120 | "\n",
121 | " while True:\n",
122 | " try:\n",
123 | " orderbook = await exchange.watch_order_book(symbol, limit = 10)\n",
124 | " orderbooks[exchange.id] = orderbooks.get(exchange.id, {})\n",
125 | " orderbooks[exchange.id][symbol] = orderbook\n",
126 | " print('===========================================================')\n",
127 | " # print(type(orderbooks), orderbooks.keys())\n",
128 | " ## code for profit calculation and order \n",
129 | "\n",
130 | " sh.post_message(json.dumps({'orderbooks': orderbooks}), channel=slack_channel)\n",
131 | " \n",
132 | " handle_all_orderbooks(orderbooks)\n",
133 | " \n",
134 | " except Exception as e:\n",
135 | " print(str(e))\n",
136 | " # raise e # uncomment to break all loops in case of an error in any one of them\n",
137 | " # break # uncomment to break just this one loop if it fails\n",
138 | " await exchange.close()"
139 | ]
140 | },
141 | {
142 | "cell_type": "code",
143 | "execution_count": 24,
144 | "id": "f516d1da-163a-48e0-b6bc-83018d36b106",
145 | "metadata": {},
146 | "outputs": [
147 | {
148 | "name": "stdout",
149 | "output_type": "stream",
150 | "text": [
151 | "symbols ['BTC/USDT', 'ETH/USDT']\n",
152 | "===========================================================\n",
153 | "None binance ETH/USDT [2587.04, 9.3855] [2587.03, 0.0386]\n",
154 | "===========================================================\n",
155 | "None binance ETH/USDT [2587.04, 9.3855] [2587.03, 0.0386]\n",
156 | "None binance BTC/USDT [37870.02, 1.86727] [37870.01, 1e-05]\n",
157 | "===========================================================\n",
158 | "2022-01-29T20:18:27.411Z binance ETH/USDT [2587.04, 9.3855] [2587.03, 0.0386]\n",
159 | "None binance BTC/USDT [37870.02, 1.86727] [37870.01, 1e-05]\n",
160 | "===========================================================\n",
161 | "2022-01-29T20:18:27.411Z binance ETH/USDT [2587.04, 9.3855] [2587.03, 0.0386]\n",
162 | "2022-01-29T20:18:27.411Z binance BTC/USDT [37870.02, 1.09138] [37870.01, 0.21132]\n"
163 | ]
164 | }
165 | ],
166 | "source": [
167 | "async def main(asyncio_loop):\n",
168 | " symbols = [f\"{t}/{base_ticker}\" for t in trade_ticker]\n",
169 | " print('symbols', symbols)\n",
170 | " exchanges = {\n",
171 | " exchange: symbols,\n",
172 | " }\n",
173 | " loops = [loop(asyncio_loop, exchange_id, symbol) for exchange_id, symbols in exchanges.items() for symbol in symbols]\n",
174 | " await asyncio.gather(*loops)\n",
175 | "\n",
176 | "\n",
177 | "if __name__ == '__main__':\n",
178 | " asyncio_loop = asyncio.get_event_loop()\n",
179 | " asyncio_loop.run_until_complete(main(asyncio_loop))"
180 | ]
181 | },
182 | {
183 | "cell_type": "code",
184 | "execution_count": null,
185 | "id": "0816123c-8d23-4cbf-a55a-58d65d54ef9b",
186 | "metadata": {},
187 | "outputs": [],
188 | "source": []
189 | }
190 | ],
191 | "metadata": {
192 | "kernelspec": {
193 | "display_name": "Python 3 (ipykernel)",
194 | "language": "python",
195 | "name": "python3"
196 | },
197 | "language_info": {
198 | "codemirror_mode": {
199 | "name": "ipython",
200 | "version": 3
201 | },
202 | "file_extension": ".py",
203 | "mimetype": "text/x-python",
204 | "name": "python",
205 | "nbconvert_exporter": "python",
206 | "pygments_lexer": "ipython3",
207 | "version": "3.8.10"
208 | }
209 | },
210 | "nbformat": 4,
211 | "nbformat_minor": 5
212 | }
213 |
--------------------------------------------------------------------------------
/example_notebooks/data_prep/rapids_data_processing.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "id": "e8facdaf",
6 | "metadata": {},
7 | "source": [
8 | "## An example of using RAPIDS to speed up pandas operations on Hyperplane\n",
9 | "- The task is to groupby and sorting about 3G of data on s3 bucket "
10 | ]
11 | },
12 | {
13 | "cell_type": "code",
14 | "execution_count": 1,
15 | "id": "af2c97bf",
16 | "metadata": {},
17 | "outputs": [],
18 | "source": [
19 | "import warnings\n",
20 | "warnings.filterwarnings(\"ignore\")"
21 | ]
22 | },
23 | {
24 | "cell_type": "code",
25 | "execution_count": 2,
26 | "id": "8cb76ab5",
27 | "metadata": {},
28 | "outputs": [],
29 | "source": [
30 | "import os\n",
31 | "import dask\n",
32 | "from hyperplane import notebook_common as nc"
33 | ]
34 | },
35 | {
36 | "cell_type": "code",
37 | "execution_count": 4,
38 | "id": "d7c4064b",
39 | "metadata": {
40 | "tags": []
41 | },
42 | "outputs": [
43 | {
44 | "name": "stdout",
45 | "output_type": "stream",
46 | "text": [
47 | "👉 Hyperplane: selecting worker node pool\n",
48 | "👉 Hyperplane: selecting scheduler node pool\n",
49 | "👉 Hyperplane: you can access your dask dashboard at https://jhub.ds.hyperplane.dev/hub/user-redirect/proxy/45601/status\n",
50 | "👉 Hyperplane: to get logs from all workers, do `cluster.get_logs()`\n"
51 | ]
52 | }
53 | ],
54 | "source": [
55 | "client, cluster = nc.initialize_cluster(\n",
56 | " nprocs=1,\n",
57 | " nthreads=8,\n",
58 | " ram_gb_per_proc=7,\n",
59 | " cores_per_worker=2,\n",
60 | " num_workers = 2,\n",
61 | " ngpus = 1,\n",
62 | " scheduler_deploy_mode=\"local\"\n",
63 | ")"
64 | ]
65 | },
66 | {
67 | "cell_type": "code",
68 | "execution_count": 5,
69 | "id": "1382c475",
70 | "metadata": {},
71 | "outputs": [
72 | {
73 | "data": {
74 | "text/html": [
75 | "\n",
78 | "Client\n", 79 | "
| \n",
84 | "\n",
85 | "Cluster\n", 86 | "
| \n",
92 | "
| \n", 177 | " | UniqueCarrier | \n", 178 | "FlightNum | \n", 179 | "Distance | \n", 180 | "
|---|---|---|---|
| 0 | \n", 185 | "PS | \n", 186 | "1451.0 | \n", 187 | "447.0 | \n", 188 | "
| 1 | \n", 191 | "PS | \n", 192 | "1451.0 | \n", 193 | "447.0 | \n", 194 | "
| 2 | \n", 197 | "PS | \n", 198 | "1451.0 | \n", 199 | "447.0 | \n", 200 | "
| 3 | \n", 203 | "PS | \n", 204 | "1451.0 | \n", 205 | "447.0 | \n", 206 | "
| 4 | \n", 209 | "PS | \n", 210 | "1451.0 | \n", 211 | "447.0 | \n", 212 | "
\n",
78 | "Client\n", 79 | "
| \n",
84 | "\n",
85 | "Cluster\n", 86 | "
| \n",
92 | "
| \n", 177 | " | UniqueCarrier | \n", 178 | "FlightNum | \n", 179 | "Distance | \n", 180 | "
|---|---|---|---|
| 0 | \n", 185 | "PS | \n", 186 | "1451.0 | \n", 187 | "447.0 | \n", 188 | "
| 1 | \n", 191 | "PS | \n", 192 | "1451.0 | \n", 193 | "447.0 | \n", 194 | "
| 2 | \n", 197 | "PS | \n", 198 | "1451.0 | \n", 199 | "447.0 | \n", 200 | "
| 3 | \n", 203 | "PS | \n", 204 | "1451.0 | \n", 205 | "447.0 | \n", 206 | "
| 4 | \n", 209 | "PS | \n", 210 | "1451.0 | \n", 211 | "447.0 | \n", 212 | "
| \n", 61 | " | class | \n", 62 | "cap-shape | \n", 63 | "cap-surface | \n", 64 | "cap-color | \n", 65 | "bruises | \n", 66 | "odor | \n", 67 | "gill-attachment | \n", 68 | "gill-spacing | \n", 69 | "gill-size | \n", 70 | "gill-color | \n", 71 | "... | \n", 72 | "stalk-surface-below-ring | \n", 73 | "stalk-color-above-ring | \n", 74 | "stalk-color-below-ring | \n", 75 | "veil-type | \n", 76 | "veil-color | \n", 77 | "ring-number | \n", 78 | "ring-type | \n", 79 | "spore-print-color | \n", 80 | "population | \n", 81 | "habitat | \n", 82 | "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", 87 | "p | \n", 88 | "x | \n", 89 | "s | \n", 90 | "n | \n", 91 | "t | \n", 92 | "p | \n", 93 | "f | \n", 94 | "c | \n", 95 | "n | \n", 96 | "k | \n", 97 | "... | \n", 98 | "s | \n", 99 | "w | \n", 100 | "w | \n", 101 | "p | \n", 102 | "w | \n", 103 | "o | \n", 104 | "p | \n", 105 | "k | \n", 106 | "s | \n", 107 | "u | \n", 108 | "
| 1 | \n", 111 | "e | \n", 112 | "x | \n", 113 | "s | \n", 114 | "y | \n", 115 | "t | \n", 116 | "a | \n", 117 | "f | \n", 118 | "c | \n", 119 | "b | \n", 120 | "k | \n", 121 | "... | \n", 122 | "s | \n", 123 | "w | \n", 124 | "w | \n", 125 | "p | \n", 126 | "w | \n", 127 | "o | \n", 128 | "p | \n", 129 | "n | \n", 130 | "n | \n", 131 | "g | \n", 132 | "
2 rows × 23 columns
\n", 136 | "| \n", 122 | " | userId | \n", 123 | "movieId | \n", 124 | "rating | \n", 125 | "timestamp | \n", 126 | "
|---|---|---|---|---|
| 0 | \n", 131 | "1 | \n", 132 | "1 | \n", 133 | "4.0 | \n", 134 | "964982703 | \n", 135 | "
| 1 | \n", 138 | "1 | \n", 139 | "3 | \n", 140 | "4.0 | \n", 141 | "964981247 | \n", 142 | "