├── .gitignore ├── README.md ├── assets ├── README.md └── project_logo.png ├── inputs └── README.md ├── models ├── README.md └── __pipeline__.ipynb ├── outputs └── README.md ├── requirements.txt ├── setup.ipynb ├── tests └── README.md └── utils ├── README.md ├── demo.ipynb └── naas.ipynb /.gitignore: -------------------------------------------------------------------------------- 1 | # Jupyter Notebook 2 | .ipynb_checkpoints 3 | 4 | # Data 5 | .csv 6 | .xlsx 7 | .json 8 | .parquet 9 | 10 | # Models 11 | pipeline_executions 12 | 13 | # Outputs 14 | outputs 15 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ℹ️ This project is currently deprecated, check the [abi repository](https://github.com/jupyter-naas/abi) to see our latest version of data product packaging. 2 | ------ 3 | 4 | 5 | ![Naas.ai - Open Source Data Platform](assets/project_logo.png) 6 | 7 | # Naas Data Product Framework 8 | 9 | Naas is a low-code open source data platform that enables anyone working with data, including business analysts, scientists, and engineers, to easily create powerful data products combining automation, analytics, and AI from the comfort of their Jupyter notebooks. With its open source distribution model, Naas ensures visible source code and versioning, and allows you to create custom logic. 10 | 11 | The platform is structured around three low-code layers: 12 | 13 | - **Templates** enable users to create automated data jobs in minutes, and are the building blocks of data products. 14 | - **Drivers** act as connectors, allowing you to push and/or pull data from databases, APIs, and machine learning algorithms, and more. 15 | - **Features** transform Jupyter notebooks into a production-ready environment, with features such as scheduling, asset sharing, and notifications. 16 | 17 | You can try Naas for free using Naas Cloud, a stable environment that runs in your browser. 18 | 19 | ## **How Does It Work?** 20 | 21 | This repository is a boilerplate for anyone who wishes to develop a data product using Naas. It is structured as follows: 22 | 23 | - The **`/assets`** folder stores any PNG, JPG, GIF, CSV, diagrams, or slides related to the documentation of the product. 24 | - The **`/inputs`** folder stores the parameters and any other files needed (data, referential) to run the files in the **`/models`** folder. 25 | - The **`/models`** folder stores any files that transform inputs into outputs (notebook, Python, SQL files) 26 | - The **`__pipeline__.ipynb`** file is used to automate the model selection process, ensuring that the most accurate model is always used for a given task. 27 | 28 | - The **`/outputs`** folder stores all the files that would be exposed outside of the Naas server. 29 | - The **`/tests`** folder stores all tests to be performed before production. 30 | - The **`/utils`** folder stores all common functions used across files. 31 | - The **`requirements.txt`** file lists all the packages and dependencies. 32 | - The **`setup.ipynb`** file runs the product on a Naas server. 33 | 34 | ## What Are The Benefits? 35 | 36 | Some benefits of the Naas Data Product Framework are: 37 | 38 | - **Low-code approach**: The low-code nature of the Naas platform makes it easy for anyone, regardless of their technical background, to create powerful data products. 39 | - **Open source**: The open source distribution model of Naas ensures visible source code and versioning, and allows you to create custom logic. 40 | - **Jupyter integration**: Naas integrates seamlessly with Jupyter notebooks, allowing you to create data products from the comfort of your familiar environment. 41 | - **Versatility**: With its templates, drivers, and features, Naas is highly versatile and enables you to build almost anything. 42 | - **Cloud-based**: Naas Cloud, the stable environment provided by Naas, allows you to access the platform from anywhere with an internet connection. 43 | 44 | Overall, the Naas Data Product Framework is a powerful tool for anyone looking to create data products that combine automation, analytics, and AI. 45 | 46 | ## Why a Data Product Development Framework Like Naas is Necessary? 47 | 48 | Just as web development frameworks like React.js help developers create web applications more efficiently by providing a set of standardized tools and components, data product development frameworks like Naas help data scientists and engineers create data products more efficiently by providing a set of standardized tools and components specifically designed for data processing, analytics, and AI. 49 | 50 | Some specific benefits of using a data product development framework like Naas include: 51 | 52 | - **Standardized structure**: A data product development framework provides a standardized structure for organizing and developing data products, which can make it easier to develop, maintain, and scale data products. 53 | - **Pre-built components**: A data product development framework includes a set of pre-built components, such as data connectors and data transformation tools, which can save time and effort compared to building these components from scratch. 54 | - **Integration with other tools**: A data product development framework typically integrates with other tools and technologies commonly used in the data world, such as Jupyter notebooks and machine learning libraries, which can make it easier to build and deploy data products. 55 | - **Collaboration and sharing**: A data product development framework can make it easier for multiple people to collaborate and share data products within an organization, as it provides a consistent framework for development and documentation. 56 | 57 | Overall, a data product development framework like Naas can provide a number of benefits to data scientists and engineers, including improved efficiency, integration with other tools, and the ability to collaborate and share data products within an organization. 58 | 59 | ## How Data Products And Asociatedd Contracts Can Create More Trust From End-User? 60 | 61 | A data product framework can help with defining data contracts and creating trust with end users in several ways: 62 | 63 | - **Standardized structure**: A data product framework provides a standardized structure for organizing and developing data products, which can make it easier to define clear and consistent data contracts. For example, if a data product is built using a framework that specifies how input and output data should be structured and documented, it can be easier for end users to understand how the data product works and what they can expect from it. 64 | - **Transparency**: Many data product frameworks are open source, which means that the source code is visible and can be reviewed by anyone. This transparency can help build trust with end users, as they can see exactly how the data product works and how it processes their data. 65 | - **Auditability**: A data product framework can also provide tools and processes for auditing and reviewing data products, which can help ensure that they are reliable and accurate. This can be especially important for data products that are used in mission-critical applications, as end users need to be confident that the data products are reliable and trustworthy. 66 | 67 | Overall, a data product framework can help create trust with end users by providing a standardized and transparent structure for developing data products, and by providing tools and processes for auditing and reviewing the products to ensure their reliability. 68 | 69 | ## **About This Repository** 70 | 71 | This Data Product Framework repository is a boilerplate to create powerful Data Products in your company. To get started: 72 | 73 | 1. Create an organization on GitHub. 74 | 2. Use this template to kickstart your Data Product. 75 | 3. Start bringing value to your company. 76 | 77 | ## **Built With** 78 | 79 | - Jupyter Notebooks 80 | - Naas 81 | 82 | ## **Documentation** 83 | 84 | ### **Prerequisites** 85 | 86 | - Create an account on naas.ai 87 | 88 | ### **Installation** 89 | 90 | Follow the steps in the **`settings.ipynb`** notebook. 91 | 92 | ## **Roadmap** 93 | 94 | - V0: Simple boilerplate with Naas pipeline feature 95 | - V1: Add Naas space feature to create powerful dashboard 96 | 97 | ## **Support** 98 | 99 | If you have problems or questions, please open an issue and we will try to help you as soon as possible. 100 | 101 | ## **Contributing** 102 | 103 | Contributions are welcome. If you have a suggestion that would make this better, please fork the repository and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star. 104 | 105 | To contribute: 106 | 107 | 1. Create an account on naas.ai. 108 | 2. Clone the repository on your machine. 109 | 3. Create a feature branch. 110 | 4. Commit your changes. 111 | 5. Push to the branch. 112 | 6. Open a pull request. 113 | 114 | 115 | ## Product Owners 116 | 117 | * [Florent Ravenel](https://www.linkedin.com/in/florent-ravenel/) - florent@naas.ai 118 | * [Jeremy Ravenel](https://www.linkedin.com/in/ACoAAAJHE7sB5OxuKHuzguZ9L6lfDHqw--cdnJg/) - jeremy@naas.ai 119 | * [Maxime Jublou](https://www.linkedin.com/in/maximejublou/) - maxime@naas.ai 120 | 121 | 122 | ## Acknowledgments 123 | 124 | * [Awesome Notebooks](https://github.com/jupyter-naas/awesome-notebooks) 125 | * [Naas Drivers](https://github.com/jupyter-naas/drivers) 126 | * [Naas](https://github.com/jupyter-naas/naas) 127 | * [Naas Data Product](https://github.com/jupyter-naas/naas-data-product) 128 | 129 | 130 | ## Legal 131 | 132 | This project is licensed under AGPL-3.0 133 | -------------------------------------------------------------------------------- /assets/README.md: -------------------------------------------------------------------------------- 1 | # Assets 2 | 3 | ## Description 4 | The /assets folder stores any PNG, JPG, GIF, CSV, diagrams, or slides related to the documentation of the product. -------------------------------------------------------------------------------- /assets/project_logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jupyter-naas/data-product-framework/bfd89550c2053b75501d9fdc11e04e0d5d7ac25a/assets/project_logo.png -------------------------------------------------------------------------------- /inputs/README.md: -------------------------------------------------------------------------------- 1 | # Inputs 2 | 3 | ## Description 4 | The /inputs folder stores the parameters and any other files needed (data, referential) to run the files in the /models folder. -------------------------------------------------------------------------------- /models/README.md: -------------------------------------------------------------------------------- 1 | # Models 2 | 3 | ## Overview 4 | The /models folder is designed to hold notebooks that can be used to quickly start working on your data product. To get started, you can navigate to the __templates__ folder and copy and paste any of the notebooks you want to use. -------------------------------------------------------------------------------- /models/__pipeline__.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "deluxe-force", 6 | "metadata": { 7 | "papermill": {}, 8 | "tags": [] 9 | }, 10 | "source": [ 11 | "\"Naas\"\n" 12 | ] 13 | }, 14 | { 15 | "cell_type": "markdown", 16 | "id": "unlimited-internship", 17 | "metadata": { 18 | "papermill": {}, 19 | "tags": [] 20 | }, 21 | "source": [ 22 | "# Naas - Create Pipeline\n", 23 | "

Template request | Bug report | Generate Data Product" 24 | ] 25 | }, 26 | { 27 | "cell_type": "markdown", 28 | "id": "3097faf3-b2b9-41fd-8589-f7718b5f919a", 29 | "metadata": { 30 | "papermill": {}, 31 | "tags": [] 32 | }, 33 | "source": [ 34 | "**Tags:** #naas #pipeline #jupyter #notebook #dataanalysis #workflow #streamline" 35 | ] 36 | }, 37 | { 38 | "cell_type": "markdown", 39 | "id": "opposite-guatemala", 40 | "metadata": { 41 | "papermill": {}, 42 | "tags": [] 43 | }, 44 | "source": [ 45 | "**Author:** [Maxime Jublou](https://www.linkedin.com/in/maximejublou)" 46 | ] 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "id": "bf2e0f26-7fdf-4351-9209-8bb54c5ef7e9", 51 | "metadata": { 52 | "papermill": {}, 53 | "tags": [] 54 | }, 55 | "source": [ 56 | "**Description:** This notebook is a guide that teaches you how to create a notebook pipeline using naas." 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "id": "input_cell", 62 | "metadata": { 63 | "papermill": {}, 64 | "tags": [] 65 | }, 66 | "source": [ 67 | "## Input" 68 | ] 69 | }, 70 | { 71 | "cell_type": "markdown", 72 | "id": "import_cell", 73 | "metadata": { 74 | "papermill": {}, 75 | "tags": [] 76 | }, 77 | "source": [ 78 | "### Import libraries" 79 | ] 80 | }, 81 | { 82 | "cell_type": "code", 83 | "execution_count": null, 84 | "id": "funny-neighbor", 85 | "metadata": { 86 | "papermill": {}, 87 | "tags": [] 88 | }, 89 | "outputs": [], 90 | "source": [ 91 | "from naas.pipeline.pipeline import (\n", 92 | " Pipeline,\n", 93 | " DummyStep,\n", 94 | " DummyErrorStep,\n", 95 | " NotebookStep,\n", 96 | " End,\n", 97 | " ParallelStep,\n", 98 | ")" 99 | ] 100 | }, 101 | { 102 | "cell_type": "markdown", 103 | "id": "0b579d95-8da2-46aa-b8dd-843ae0d46964", 104 | "metadata": {}, 105 | "source": [ 106 | "### Setup variables\n", 107 | "- `pipeline_outputs_path`: Path of your pipeline executions. When the pipeline is run, a \"pipeline_executions\" folder will be created in your file system. Inside this folder, you will be able to access each pipeline executions. If you use NotebookStep, executed notebooks will be stored in this folder. This allows you to easily review and analyze the results of the pipeline, and to troubleshoot any issues that may have occurred." 108 | ] 109 | }, 110 | { 111 | "cell_type": "code", 112 | "execution_count": null, 113 | "id": "f67191c1-aaa8-4ad6-8793-eb563f72822c", 114 | "metadata": {}, 115 | "outputs": [], 116 | "source": [ 117 | "pipeline_outputs_path = \"../outputs/pipeline_executions\"" 118 | ] 119 | }, 120 | { 121 | "cell_type": "markdown", 122 | "id": "model_cell", 123 | "metadata": { 124 | "papermill": {}, 125 | "tags": [] 126 | }, 127 | "source": [ 128 | "## Model" 129 | ] 130 | }, 131 | { 132 | "cell_type": "markdown", 133 | "id": "e15d093c-0189-42c3-959f-64eed5530fd4", 134 | "metadata": { 135 | "papermill": {}, 136 | "tags": [] 137 | }, 138 | "source": [ 139 | "### Setup NotebookStep\n", 140 | "For demonstration purposes, we used the `DummyStep` to illustrate its functionality.\n", 141 | "\n", 142 | "In order to create a pipeline, you should used the `NotebookStep()` which has three parameters:\n", 143 | "- the name (string) of the step\n", 144 | "- the notebook path (string) for execution\n", 145 | "- the parameters (dictionary) that are injected through papermill in the first cell or after the cell labeled \"parameters.\"\n", 146 | "\n", 147 | "`NotebookStep(\"My Notebook\", \"my_notebook.ipynb\")`" 148 | ] 149 | }, 150 | { 151 | "cell_type": "code", 152 | "execution_count": null, 153 | "id": "fa346868-d375-4547-b11b-79311e3f1fc3", 154 | "metadata": { 155 | "papermill": {}, 156 | "tags": [] 157 | }, 158 | "outputs": [], 159 | "source": [ 160 | "collection = DummyStep(\n", 161 | " \"Collection\"\n", 162 | ") # In this step, data can be collected from various sources such as databases, APIs, or file systems.\n", 163 | "cleaning = DummyStep(\n", 164 | " \"Cleaning\"\n", 165 | ") # Once the data is collected, it is often necessary to clean and preprocess it to remove any irrelevant or duplicate information. This step may involve tasks such as removing null values, correcting data formats, and standardizing column names.\n", 166 | "transformation1 = DummyStep(\n", 167 | " \"Transformation 1\"\n", 168 | ") # In this step, the data is transformed into the desired format, such as a flat file or a specific data model. This may involve tasks such as aggregating data, joining multiple tables, or calculating new fields.\n", 169 | "transformation2 = DummyStep(\n", 170 | " \"Transformation 2\"\n", 171 | ") # In this step, the data is transformed into the desired format, such as a flat file or a specific data model. This may involve tasks such as aggregating data, joining multiple tables, or calculating new fields.\n", 172 | "distribution = DummyStep(\n", 173 | " \"Distribution\"\n", 174 | ") # In this step, the data is loaded into its final destination, such as a data warehouse, a data lake, or a specific application." 175 | ] 176 | }, 177 | { 178 | "cell_type": "markdown", 179 | "id": "output_cell", 180 | "metadata": { 181 | "papermill": {}, 182 | "tags": [] 183 | }, 184 | "source": [ 185 | "## Output" 186 | ] 187 | }, 188 | { 189 | "cell_type": "markdown", 190 | "id": "display_cell", 191 | "metadata": { 192 | "papermill": {}, 193 | "tags": [] 194 | }, 195 | "source": [ 196 | "### Create Basic Pipeline\n", 197 | "- Link your notebook using this syntax: `>>`\n", 198 | "- Create ParallelStep using this syntax: `[step1, step2]`" 199 | ] 200 | }, 201 | { 202 | "cell_type": "code", 203 | "execution_count": null, 204 | "id": "c14e4a38-75b1-475b-bc28-45db7effe45f", 205 | "metadata": { 206 | "papermill": {}, 207 | "tags": [] 208 | }, 209 | "outputs": [], 210 | "source": [ 211 | "pipeline = Pipeline()\n", 212 | "\n", 213 | "(\n", 214 | " pipeline\n", 215 | " >> collection\n", 216 | " >> cleaning\n", 217 | " >> [transformation1, transformation2]\n", 218 | " >> distribution\n", 219 | " >> End()\n", 220 | ")\n", 221 | "\n", 222 | "pipeline.run(outputs_path=pipeline_outputs_path)" 223 | ] 224 | } 225 | ], 226 | "metadata": { 227 | "kernelspec": { 228 | "display_name": "Python 3", 229 | "language": "python", 230 | "name": "python3" 231 | }, 232 | "language_info": { 233 | "codemirror_mode": { 234 | "name": "ipython", 235 | "version": 3 236 | }, 237 | "file_extension": ".py", 238 | "mimetype": "text/x-python", 239 | "name": "python", 240 | "nbconvert_exporter": "python", 241 | "pygments_lexer": "ipython3", 242 | "version": "3.9.6" 243 | }, 244 | "naas": { 245 | "notebook_id": "92ddbcf7c74813cc4c906ca6b7d04cc2590230b5fb16082b396de5b9872be0cf", 246 | "notebook_path": "Naas/Naas_Create_Pipeline.ipynb" 247 | }, 248 | "papermill": { 249 | "default_parameters": {}, 250 | "environment_variables": {}, 251 | "parameters": {}, 252 | "version": "2.3.3" 253 | } 254 | }, 255 | "nbformat": 4, 256 | "nbformat_minor": 5 257 | } 258 | -------------------------------------------------------------------------------- /outputs/README.md: -------------------------------------------------------------------------------- 1 | # Outputs 2 | 3 | ## Description 4 | The /outputs folder stores all the files that would be exposed outside of the Naas server -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | naas_data_product 2 | pyvis==0.2.1 -------------------------------------------------------------------------------- /setup.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "injured-evolution", 6 | "metadata": { 7 | "papermill": {}, 8 | "tags": [] 9 | }, 10 | "source": [ 11 | "\"Naas\"" 12 | ] 13 | }, 14 | { 15 | "cell_type": "markdown", 16 | "id": "substantial-decline", 17 | "metadata": { 18 | "papermill": {}, 19 | "tags": [] 20 | }, 21 | "source": [ 22 | "# Settings" 23 | ] 24 | }, 25 | { 26 | "cell_type": "markdown", 27 | "id": "d288008a-adf4-47e8-acf5-55a0d2f9ae4e", 28 | "metadata": {}, 29 | "source": [ 30 | "
\n", 31 | "

Note: this data product framework is developed by Naas open source community. You can sponsor us if you find it usefull.\n", 32 | "

\n", 33 | "
" 34 | ] 35 | }, 36 | { 37 | "cell_type": "markdown", 38 | "id": "dd3d4d6a-f15d-47f9-83ac-4208edf13bf9", 39 | "metadata": { 40 | "tags": [] 41 | }, 42 | "source": [ 43 | "This notebook allows you to configure and run your data product on your Naas server. *Create an account on [Naas.ai](https://www.naas.ai/free-forever) if you don't already have one.*\n", 44 | "\n", 45 | "Please follow the step below to setup this data product." 46 | ] 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "id": "80056fe3-734f-41fe-8709-6838316e2f89", 51 | "metadata": {}, 52 | "source": [ 53 | "## Input" 54 | ] 55 | }, 56 | { 57 | "cell_type": "markdown", 58 | "id": "4df006e1-c200-4a11-9b37-473864001e67", 59 | "metadata": {}, 60 | "source": [ 61 | "### Install requirements\n", 62 | "Install the packages required to run the product." 63 | ] 64 | }, 65 | { 66 | "cell_type": "code", 67 | "execution_count": null, 68 | "id": "d7e9b5d5-36ff-4ba2-bce8-253367a73949", 69 | "metadata": { 70 | "tags": [] 71 | }, 72 | "outputs": [], 73 | "source": [ 74 | "!pip install --user --upgrade -r \"requirements.txt\"" 75 | ] 76 | }, 77 | { 78 | "cell_type": "markdown", 79 | "id": "d3bba624-afe5-484b-a5d3-0d4bf7dd019c", 80 | "metadata": {}, 81 | "source": [ 82 | "### Import libraries" 83 | ] 84 | }, 85 | { 86 | "cell_type": "code", 87 | "execution_count": null, 88 | "id": "21d997a6-cfee-468d-8009-53657d604ae7", 89 | "metadata": { 90 | "tags": [] 91 | }, 92 | "outputs": [], 93 | "source": [ 94 | "import naas" 95 | ] 96 | }, 97 | { 98 | "cell_type": "markdown", 99 | "id": "7e4860ba-61b0-4dc7-b290-3ba30b328b6b", 100 | "metadata": {}, 101 | "source": [ 102 | "### Setup variables\n", 103 | "- `pipeline_path`: pipeline path\n", 104 | "- `cron`: cron params for naas scheduler\n", 105 | "- `dependencies`: list of files (path) that must be sent to production when your pipeline is executed" 106 | ] 107 | }, 108 | { 109 | "cell_type": "code", 110 | "execution_count": null, 111 | "id": "6ef71640-f789-4331-bba1-b5def9d6920c", 112 | "metadata": { 113 | "tags": [] 114 | }, 115 | "outputs": [], 116 | "source": [ 117 | "pipeline_path = f\"models/__pipeline__.ipynb\"\n", 118 | "cron = \"0 8 * * *\"\n", 119 | "dependencies = []" 120 | ] 121 | }, 122 | { 123 | "cell_type": "markdown", 124 | "id": "2ebb82c4-a40a-496e-a1fd-2c04e39455bd", 125 | "metadata": {}, 126 | "source": [ 127 | "## Model" 128 | ] 129 | }, 130 | { 131 | "cell_type": "markdown", 132 | "id": "2373f75e-9c82-4a1d-a0df-3bc5cf3788e8", 133 | "metadata": {}, 134 | "source": [ 135 | "### Schedule notebook" 136 | ] 137 | }, 138 | { 139 | "cell_type": "code", 140 | "execution_count": null, 141 | "id": "db6ac468-b33e-499f-a87b-16827d0a5beb", 142 | "metadata": { 143 | "tags": [] 144 | }, 145 | "outputs": [], 146 | "source": [ 147 | "naas.scheduler.add(pipeline_path, cron=cron)\n", 148 | "\n", 149 | "# Delete scheduler\n", 150 | "# naas.scheduler.delete(pipeline_path)" 151 | ] 152 | }, 153 | { 154 | "cell_type": "markdown", 155 | "id": "5bc270c0-60e5-479b-97fc-06e11feaadcb", 156 | "metadata": {}, 157 | "source": [ 158 | "### Send dependencies to production" 159 | ] 160 | }, 161 | { 162 | "cell_type": "code", 163 | "execution_count": null, 164 | "id": "ea9dfb8a-675c-4fc2-973b-f1b6fcbb9f04", 165 | "metadata": { 166 | "tags": [] 167 | }, 168 | "outputs": [], 169 | "source": [ 170 | "for dependency in dependencies:\n", 171 | " naas.dependency.add(dependency)" 172 | ] 173 | }, 174 | { 175 | "cell_type": "markdown", 176 | "id": "486ca79e-21af-47c8-ae9a-1c4d1d7cf310", 177 | "metadata": {}, 178 | "source": [ 179 | "## Output" 180 | ] 181 | } 182 | ], 183 | "metadata": { 184 | "kernelspec": { 185 | "display_name": "Python 3", 186 | "language": "python", 187 | "name": "python3" 188 | }, 189 | "language_info": { 190 | "codemirror_mode": { 191 | "name": "ipython", 192 | "version": 3 193 | }, 194 | "file_extension": ".py", 195 | "mimetype": "text/x-python", 196 | "name": "python", 197 | "nbconvert_exporter": "python", 198 | "pygments_lexer": "ipython3", 199 | "version": "3.9.6" 200 | }, 201 | "naas": { 202 | "notebook_id": "77b7f93a-fb2a-483e-b98b-80735e426a64", 203 | "notebook_path": "Naas/Naas_Secret_demo.ipynb" 204 | }, 205 | "papermill": { 206 | "default_parameters": {}, 207 | "environment_variables": {}, 208 | "parameters": {}, 209 | "version": "2.3.3" 210 | }, 211 | "toc-autonumbering": false, 212 | "widgets": { 213 | "application/vnd.jupyter.widget-state+json": { 214 | "state": {}, 215 | "version_major": 2, 216 | "version_minor": 0 217 | } 218 | } 219 | }, 220 | "nbformat": 4, 221 | "nbformat_minor": 5 222 | } 223 | -------------------------------------------------------------------------------- /tests/README.md: -------------------------------------------------------------------------------- 1 | # Tests 2 | 3 | ## Description 4 | The /tests folder stores all tests to be performed before production. 5 | -------------------------------------------------------------------------------- /utils/README.md: -------------------------------------------------------------------------------- 1 | # Utils 2 | 3 | ## Description 4 | The /utils folder stores all common functions used across files. 5 | 6 | 7 | -------------------------------------------------------------------------------- /utils/demo.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "0fe69480-22c1-4927-9217-6bdb459b5d56", 6 | "metadata": {}, 7 | "source": [ 8 | "\"Naas\"" 9 | ] 10 | }, 11 | { 12 | "cell_type": "markdown", 13 | "id": "1da2bba4-1b4c-4daa-a7c8-487c3a740eba", 14 | "metadata": {}, 15 | "source": [ 16 | "# Demo" 17 | ] 18 | }, 19 | { 20 | "cell_type": "markdown", 21 | "id": "a03e20c0-49a0-4204-b5f9-c1f89850dd8b", 22 | "metadata": {}, 23 | "source": [ 24 | "## Input" 25 | ] 26 | }, 27 | { 28 | "cell_type": "markdown", 29 | "id": "dbc31862-a80f-4f73-b41c-45c616c7991b", 30 | "metadata": {}, 31 | "source": [ 32 | "### Import libraries" 33 | ] 34 | }, 35 | { 36 | "cell_type": "code", 37 | "execution_count": null, 38 | "id": "ee961415-6782-4a06-96e8-f6ab708631a3", 39 | "metadata": { 40 | "tags": [] 41 | }, 42 | "outputs": [], 43 | "source": [] 44 | }, 45 | { 46 | "cell_type": "markdown", 47 | "id": "17772827-ea8b-4d47-be40-8be22d3a6568", 48 | "metadata": {}, 49 | "source": [ 50 | "## Model" 51 | ] 52 | }, 53 | { 54 | "cell_type": "markdown", 55 | "id": "7d004a5c-6ef5-4221-a487-bb095bb3d6dd", 56 | "metadata": {}, 57 | "source": [ 58 | "### Common function" 59 | ] 60 | }, 61 | { 62 | "cell_type": "code", 63 | "execution_count": null, 64 | "id": "bbae52dc-1f25-462b-b348-57ba1ad7a9f0", 65 | "metadata": {}, 66 | "outputs": [], 67 | "source": [] 68 | }, 69 | { 70 | "cell_type": "markdown", 71 | "id": "c56e7467-89b1-4d83-9633-96453709a143", 72 | "metadata": {}, 73 | "source": [ 74 | "## Output" 75 | ] 76 | }, 77 | { 78 | "cell_type": "markdown", 79 | "id": "59b5128a-6d43-4bbb-9c0d-bd5c93833220", 80 | "metadata": {}, 81 | "source": [ 82 | "### Display output" 83 | ] 84 | }, 85 | { 86 | "cell_type": "code", 87 | "execution_count": null, 88 | "id": "bf57582a-5844-49d9-a1b6-eb9f61db2e8b", 89 | "metadata": { 90 | "tags": [] 91 | }, 92 | "outputs": [], 93 | "source": [ 94 | "print(f\"✅ Demo loaded successfully.\")" 95 | ] 96 | } 97 | ], 98 | "metadata": { 99 | "kernelspec": { 100 | "display_name": "Python 3", 101 | "language": "python", 102 | "name": "python3" 103 | }, 104 | "language_info": { 105 | "codemirror_mode": { 106 | "name": "ipython", 107 | "version": 3 108 | }, 109 | "file_extension": ".py", 110 | "mimetype": "text/x-python", 111 | "name": "python", 112 | "nbconvert_exporter": "python", 113 | "pygments_lexer": "ipython3", 114 | "version": "3.9.6" 115 | }, 116 | "widgets": { 117 | "application/vnd.jupyter.widget-state+json": { 118 | "state": {}, 119 | "version_major": 2, 120 | "version_minor": 0 121 | } 122 | } 123 | }, 124 | "nbformat": 4, 125 | "nbformat_minor": 5 126 | } 127 | -------------------------------------------------------------------------------- /utils/naas.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "id": "0fe69480-22c1-4927-9217-6bdb459b5d56", 6 | "metadata": {}, 7 | "source": [ 8 | "\"Naas\"" 9 | ] 10 | }, 11 | { 12 | "cell_type": "markdown", 13 | "id": "1da2bba4-1b4c-4daa-a7c8-487c3a740eba", 14 | "metadata": {}, 15 | "source": [ 16 | "# Naas" 17 | ] 18 | }, 19 | { 20 | "cell_type": "markdown", 21 | "id": "a03e20c0-49a0-4204-b5f9-c1f89850dd8b", 22 | "metadata": {}, 23 | "source": [ 24 | "## Input" 25 | ] 26 | }, 27 | { 28 | "cell_type": "markdown", 29 | "id": "dbc31862-a80f-4f73-b41c-45c616c7991b", 30 | "metadata": {}, 31 | "source": [ 32 | "### Import libraries" 33 | ] 34 | }, 35 | { 36 | "cell_type": "code", 37 | "execution_count": 2, 38 | "id": "ee961415-6782-4a06-96e8-f6ab708631a3", 39 | "metadata": { 40 | "execution": { 41 | "iopub.execute_input": "2023-05-04T13:37:42.474884Z", 42 | "iopub.status.busy": "2023-05-04T13:37:42.474649Z", 43 | "iopub.status.idle": "2023-05-04T13:37:42.482384Z", 44 | "shell.execute_reply": "2023-05-04T13:37:42.481812Z", 45 | "shell.execute_reply.started": "2023-05-04T13:37:42.474861Z" 46 | }, 47 | "tags": [] 48 | }, 49 | "outputs": [], 50 | "source": [ 51 | "import naas" 52 | ] 53 | }, 54 | { 55 | "cell_type": "markdown", 56 | "id": "17772827-ea8b-4d47-be40-8be22d3a6568", 57 | "metadata": {}, 58 | "source": [ 59 | "## Model" 60 | ] 61 | }, 62 | { 63 | "cell_type": "markdown", 64 | "id": "af62b007-b93f-47a9-9019-5cc2f67fe959", 65 | "metadata": {}, 66 | "source": [ 67 | "### Setup secrets from list" 68 | ] 69 | }, 70 | { 71 | "cell_type": "code", 72 | "execution_count": null, 73 | "id": "29dcab2b-b969-4fa6-91ef-632ed1879b7d", 74 | "metadata": { 75 | "tags": [] 76 | }, 77 | "outputs": [], 78 | "source": [ 79 | "def setup_secrets_from_list(list_secrets_name, force_update=False):\n", 80 | " for secret in list_secrets_name:\n", 81 | " secret_value = naas.secret.get(secret)\n", 82 | " if not secret_value or force_update:\n", 83 | " print(\"🔑 Setup secret:\", secret)\n", 84 | " secret_value = input(\"Secret value:\")\n", 85 | " if secret_value:\n", 86 | " naas.secret.add(secret, secret_value)" 87 | ] 88 | }, 89 | { 90 | "cell_type": "markdown", 91 | "id": "c56e7467-89b1-4d83-9633-96453709a143", 92 | "metadata": {}, 93 | "source": [ 94 | "## Output" 95 | ] 96 | } 97 | ], 98 | "metadata": { 99 | "kernelspec": { 100 | "display_name": "Python 3", 101 | "language": "python", 102 | "name": "python3" 103 | }, 104 | "language_info": { 105 | "codemirror_mode": { 106 | "name": "ipython", 107 | "version": 3 108 | }, 109 | "file_extension": ".py", 110 | "mimetype": "text/x-python", 111 | "name": "python", 112 | "nbconvert_exporter": "python", 113 | "pygments_lexer": "ipython3", 114 | "version": "3.9.6" 115 | }, 116 | "widgets": { 117 | "application/vnd.jupyter.widget-state+json": { 118 | "state": {}, 119 | "version_major": 2, 120 | "version_minor": 0 121 | } 122 | } 123 | }, 124 | "nbformat": 4, 125 | "nbformat_minor": 5 126 | } 127 | --------------------------------------------------------------------------------