├── .github ├── CODEOWNERS ├── ISSUE_TEMPLATE │ ├── config.yaml │ ├── feature_request.yaml │ └── report_bug.yaml ├── PULL_REQUEST_TEMPLATE.md └── workflows │ └── pypi-release.yaml ├── .gitignore ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── MANIFEST.in ├── README.md ├── SECURITY.md ├── __init__.py ├── _config.yml ├── dashboards └── grafana_langchain.json ├── demo.py ├── demo ├── __init__.py ├── fakechat_langchain.py ├── fastapi_langchain │ ├── __init__.py │ ├── local.sh │ ├── main.py │ └── requirements.txt ├── mockgpt_langchain.py ├── mockgpt_runnable_langchain.py └── openai_langchain.py ├── demo_requirements.txt ├── docs ├── images │ └── langchain │ │ ├── langchain-dashboard-1.png │ │ ├── langchain-dashboard-2.png │ │ ├── langchain-dashboard-3.png │ │ └── langchain-dashboard-4.png └── langchain.md ├── requirements.txt ├── requirements ├── base_requirements.txt └── requirements_langchain.txt ├── setup.py └── vishwa ├── __init__.py ├── client ├── __init__.py ├── constants.py ├── models.py └── xpuls_client.py ├── mlmonitor ├── __init__.py ├── langchain │ ├── __init__.py │ ├── decorators │ │ ├── __init__.py │ │ ├── map_xpuls_project.py │ │ └── telemetry_override_labels.py │ ├── handlers │ │ ├── __init__.py │ │ ├── callback_handlers.py │ │ └── constants.py │ ├── instrument.py │ ├── patches │ │ ├── __init__.py │ │ ├── patch_invoke.py │ │ ├── patch_run.py │ │ ├── utils.py │ │ └── xp_prompt_template.py │ ├── profiling │ │ ├── __init__.py │ │ └── prometheus.py │ └── xpuls_client.py └── utils │ ├── __init__.py │ └── common.py └── prompt_hub ├── __init__.py └── prompt.py /.github/CODEOWNERS: -------------------------------------------------------------------------------- 1 | * SHARANTANGEDA 2 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/config.yaml: -------------------------------------------------------------------------------- 1 | blank_issues_enabled: true 2 | version: 2.1 3 | contact_links: 4 | - name: Reach out to vishwa.ai 5 | url: https://vishwa.ai 6 | about: For more information about idea and people behind this OSS project. 7 | - name: Vishwa Labs Discussions 8 | url: https://github.com/vishwa-labs/vishwa-ml-sdk/discussions 9 | about: Please ask general questions here. 10 | - name: Reachout to owner 11 | url: sharan@vishwa.tech 12 | about: Contact the maintainer for catchup or support 13 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/feature_request.yaml: -------------------------------------------------------------------------------- 1 | name: Request a new Feature (🚀) 2 | description: Submit a proposal/request for new a feature or new framework. 3 | title: "feature: " 4 | labels: ["new-feature", "enhancement"] 5 | body: 6 | - type: textarea 7 | id: feature-request 8 | validations: 9 | required: true 10 | attributes: 11 | label: Feature request 12 | description: | 13 | A clear and concise description of the feature request. 14 | placeholder: | 15 | I would like it if... 16 | - type: textarea 17 | id: motivation 18 | validations: 19 | required: false 20 | attributes: 21 | label: Motivation 22 | description: | 23 | Please outline the motivation for this feature request. 24 | Is your feature request related to a problem? e.g., I'm always frustrated when [...]. 25 | Or Is this about lack of support for any ML Frameworks? 26 | If this is related to another issue, please link here too. 27 | If you have a current workaround, please also provide it here. 28 | placeholder: | 29 | This feature would solve ... 30 | - type: textarea 31 | id: other 32 | attributes: 33 | label: Other 34 | description: | 35 | Is there any way that you could help, e.g. by submitting a PR?. 36 | placeholder: | 37 | I would love to contribute ... 38 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/report_bug.yaml: -------------------------------------------------------------------------------- 1 | name: Report a bug (🐛) 2 | description: Create a bug report to help us improve vishwa-ml-sdk 3 | title: "bug: " 4 | labels: ["bug"] 5 | body: 6 | - type: markdown 7 | id: issue-already-exists 8 | attributes: 9 | value: | 10 | Please search to see if an issue already exists for the bug you encountered. 11 | See [Searching Issues and Pull Requests](https://docs.github.com/en/search-github/searching-on-github/searching-issues-and-pull-requests) for how to use the GitHub search bar and filters. 12 | - type: textarea 13 | id: describe-the-bug 14 | validations: 15 | required: true 16 | attributes: 17 | label: Describe the bug 18 | description: Please provide a clear and concise description about the problem you ran into. 19 | placeholder: This happened when I... 20 | - type: textarea 21 | id: to-reproduce 22 | validations: 23 | required: false 24 | attributes: 25 | label: To reproduce 26 | description: | 27 | Please provide a code sample or a code snippet to reproduce said problem. If you have code snippets, error messages, stack trace please also provide them here. 28 | 29 | **IMPORTANT**: make sure to use [code tag](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/creating-and-highlighting-code-blocks#syntax-highlighting) to correctly format your code. 30 | Screenshot is helpful but don't use it for code snippets as it doesn't allow others to copy-and-paste your code. 31 | 32 | To give us more information for diagnosing the issue, make sure to enable debug logging: 33 | 34 | Set debug logging in your Python code: 35 | ```python 36 | import logging 37 | 38 | app_logger = logging.getLogger() 39 | app_logger.setLevel(logging.DEBUG) 40 | ``` 41 | 42 | placeholder: | 43 | Steps to reproduce the bug: 44 | 45 | 1. Provide '...' 46 | 2. Run '...' 47 | 3. See error 48 | - type: textarea 49 | id: expected-behavior 50 | validations: 51 | required: false 52 | attributes: 53 | label: Expected behavior 54 | description: "A clear and concise description of what you would expect to happen." 55 | -------------------------------------------------------------------------------- /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | ## What does this PR address? 2 | 3 | 4 | Thanks for sending a pull request! 5 | 6 | 29 | 30 | 31 | 32 | Fixes #(issue) 33 | 34 | ## Before submitting: 35 | 36 | 37 | 38 | 39 | 40 | - [ ] Does the Pull Request follow [Conventional Commits specification](https://www.conventionalcommits.org/en/v1.0.0/#summary) naming? Here are [GitHub's 41 | guide](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request) on how to create a pull request. 42 | - [ ] Does the code follow mlmonitor's code style, `pre-commit run -a` script has passed ? 43 | - [ ] Did your changes require updates to the documentation? Have you updated 44 | those accordingly? 45 | - [ ] Did you write tests to cover your changes? -------------------------------------------------------------------------------- /.github/workflows/pypi-release.yaml: -------------------------------------------------------------------------------- 1 | name: pypi-release 2 | on: 3 | push: 4 | tags: 5 | - v* 6 | jobs: 7 | build: 8 | runs-on: ubuntu-latest 9 | steps: 10 | - uses: actions/checkout@v3 11 | - name: Setup Python 12 | uses: actions/setup-python@v4 13 | with: 14 | python-version: '3.11' 15 | - name: Install dependencies 16 | run: python -m pip install --upgrade build 17 | - name: Build 18 | run: python -m build 19 | - name: Publish to PyPI 20 | uses: pypa/gh-action-pypi-publish@release/v1 21 | with: 22 | password: ${{ secrets.PYPI_PUBLIC_TOKEN }} 23 | - name: Archive 24 | uses: actions/upload-artifact@v3 25 | with: 26 | name: dist 27 | path: dist/* 28 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .idea 2 | venv 3 | dist 4 | 5 | *.env 6 | -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as 6 | contributors and maintainers pledge to making participation in our project and 7 | our community a harassment-free experience for everyone, regardless of age, body 8 | size, disability, ethnicity, sex characteristics, gender identity and expression, 9 | level of experience, education, socio-economic status, nationality, personal 10 | appearance, race, religion, or sexual identity and orientation. 11 | 12 | ## Our Standards 13 | 14 | Examples of behavior that contributes to creating a positive environment 15 | include: 16 | 17 | * Using welcoming and inclusive language 18 | * Being respectful of differing viewpoints and experiences 19 | * Gracefully accepting constructive criticism 20 | * Focusing on what is best for the community 21 | * Showing empathy towards other community members 22 | 23 | Examples of unacceptable behavior by participants include: 24 | 25 | * The use of sexualized language or imagery and unwelcome sexual attention or 26 | advances 27 | * Trolling, insulting/derogatory comments, and personal or political attacks 28 | * Public or private harassment 29 | * Publishing others' private information, such as a physical or electronic 30 | address, without explicit permission 31 | * Other conduct which could reasonably be considered inappropriate in a 32 | professional setting 33 | 34 | ## Our Responsibilities 35 | 36 | Project maintainers are responsible for clarifying the standards of acceptable 37 | behavior and are expected to take appropriate and fair corrective action in 38 | response to any instances of unacceptable behavior. 39 | 40 | Project maintainers have the right and responsibility to remove, edit, or 41 | reject comments, commits, code, wiki edits, issues, and other contributions 42 | that are not aligned to this Code of Conduct, or to ban temporarily or 43 | permanently any contributor for other behaviors that they deem inappropriate, 44 | threatening, offensive, or harmful. 45 | 46 | ## Scope 47 | 48 | This Code of Conduct applies both within project spaces and in public spaces 49 | when an individual is representing the project or its community. Examples of 50 | representing a project or community include using an official project e-mail 51 | address, posting via an official social media account, or acting as an appointed 52 | representative at an online or offline event. Representation of a project may be 53 | further defined and clarified by project maintainers. 54 | 55 | ## Enforcement 56 | 57 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 58 | reported by contacting the support team at support@vishwa.ai. All 59 | complaints will be reviewed and investigated and will result in a response that 60 | is deemed necessary and appropriate to the circumstances. The support team is 61 | obligated to maintain confidentiality with regard to the reporter of an incident. 62 | Further details of specific enforcement policies may be posted separately. 63 | 64 | Project maintainers who do not follow or enforce the Code of Conduct in good 65 | faith may face temporary or permanent repercussions as determined by other 66 | members of the vishwa.ai's leadership. 67 | 68 | ## Attribution 69 | 70 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, 71 | available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html 72 | 73 | [homepage]: https://www.contributor-covenant.org 74 | 75 | For answers to common questions about this code of conduct, see 76 | https://www.contributor-covenant.org/faq 77 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contribution Guidelines 2 | 3 | ## Welcome to vishwa.ai Contribution guide. 4 | 5 | We are super excited to see that you'd like to contribute to our platform, thanks for your interest. 6 | 7 | We value any feedback, bug report or feature requests raised by our community and are determined to solving them. 8 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright (c) 2023 - Cognicraft Technologies 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include requirements/base_requirements.txt 2 | include requirements.txt 3 | include requirements/requirements_langchain.txt 4 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Welcome to vishwa.ai (formerly xpuls.ai) 👋 2 | 3 | ## vishwa-ml-sdk 4 | [![Twitter Follow](https://img.shields.io/twitter/follow/vishwa_ai?style=social)](https://x.com/vishwa_ai) [![Discord](https://img.shields.io/badge/Discord-Join-1147943825592045689?style=social)](https://social.vishwa.ai/join/discord) 5 | 6 | 7 | 8 |
9 | Website | Docs | News | Twitter | Discord 10 |
11 | 12 | [![PyPI version](https://badge.fury.io/py/vishwa-ml-sdk.svg)](https://badge.fury.io/py/vishwa-ml-sdk) 13 | [![GitHub version](https://badge.fury.io/gh/vishwa-labs%2Fvishwa-ml-sdk.svg)](https://badge.fury.io/gh/vishwa-labs%2Fvishwa-ml-sdk) 14 | 15 | ## Roadmap 🚀 16 | 17 | | Framework | Status | 18 | |------------------|---------| 19 | | Langchain | ✅ | 20 | | LLamaIndex | Planned | 21 | | PyTorch | Planned | 22 | | SKLearn | Planned | 23 | | Transformers | Planned | 24 | | Stable Diffusion | Next | 25 | 26 | 27 | ### 💡 If support of any framework/feature is useful for you, please feel free to reach out to us via [Discord](https://social.vishwa.ai/join/discord) or Github Discussions 28 | 29 | 30 | ## 🔗 Installation 31 | 32 | 1. Install from PyPI 33 | ```shell 34 | pip install vishwa-ml-sdk 35 | ``` 36 | 37 | ## 🧩 Usage Example 38 | ```python 39 | from vishwa.mlmonitor.langchain.instrument import LangchainTelemetry 40 | import os 41 | import vishwa 42 | from vishwa.prompt_hub import PromptClient 43 | 44 | # Enable this for advance tracking with our vishwa-ai platform 45 | vishwa.host_url = "https://api.vishwa.ai" 46 | vishwa.api_key = "********************" # Get from https://platform.vishwa.ai 47 | vishwa.adv_tracing_enabled = "true" # Enable this for automated insights and log tracing via xpulsAI platform 48 | # Add default labels that will be added to all captured metrics 49 | default_labels = {"service": "ml-project-service", "k8s_cluster": "app0", "namespace": "dev", "agent_name": "fallback_value"} 50 | 51 | # Enable the auto-telemetry 52 | LangchainTelemetry(default_labels=default_labels,).auto_instrument() 53 | prompt_client = PromptClient( 54 | prompt_id="clrfm4v70jnlb1kph240", # Get prompt_id from the platform 55 | environment_name="dev" # Deployed environment name 56 | ) 57 | 58 | ## [Optional] Override labels for scope of decorator [Useful if you have multiple scopes where you need to override the default label values] 59 | @TelemetryOverrideLabels(agent_name="chat_agent_alpha") 60 | @TagToProject(project_slug="defaultoPIt9USSR") # Get Project Slug from platform 61 | def get_response_using_agent_alpha(prompt, query): 62 | agent = initialize_agent(llm=chat_model, 63 | verbose=True, 64 | agent=CONVERSATIONAL_REACT_DESCRIPTION, 65 | memory=memory) 66 | 67 | data = prompt_client.get_prompt({"variable-1": "I'm the first variable"}) # Substitute any variables in prompt 68 | 69 | res = agent.run(data) # Pass the entire `XPPrompt` object to run or invoke method 70 | ``` 71 | 72 | ## ℹ️ Complete Usage Guides 73 | 74 | - [Langchain Framework](./docs/langchain.md) + [Grafana Template](./dashboards/grafana_langchain.json) 75 | 76 | ## 🧾 License 77 | 78 | This project is licensed under the Apache License 2.0. See the LICENSE file for more details. 79 | 80 | 81 | ## 📢 Contributing 82 | 83 | We welcome contributions to xpuls-ml-sdk! If you're interested in contributing. 84 | 85 | If you encounter any issues or have feature requests, please file an issue on our GitHub repository. 86 | 87 | 88 | ## 💬 Get in touch 89 | 90 | 👉 [Join our Discord community!](https://social.vishwa.ai/join/discord) 91 | 92 | 🐦 Follow the latest from vishwa.ai team on Twitter [@vishwa_ai](https://twitter.com/vishwa_ai) 93 | 94 | 📮 Write to us at [hello\@vishwa.ai](mailto:hello@vishwa.ai) 95 | -------------------------------------------------------------------------------- /SECURITY.md: -------------------------------------------------------------------------------- 1 | # Security Policy 2 | 3 | xpxuls.ai is looking forward to working with security researchers across the world to keep vishwa-ml-sdk along with other products and our users safe. If you have found an issue in our systems/applications, please reach out to us. 4 | 5 | ## Supported Versions 6 | We always recommend using the latest version of vishwa.ai to ensure you get all security updates 7 | 8 | ## Reporting a Vulnerability 9 | 10 | If you believe you have found a security vulnerability within vishwa.ai, please let us know right away. We'll try and fix the problem as soon as possible. 11 | 12 | **Do not report vulnerabilities using public GitHub issues**. Instead, email with a detailed account of the issue. Please submit one issue per email, this helps us triage vulnerabilities. 13 | 14 | Once we've received your email we'll keep you updated as we fix the vulnerability. 15 | 16 | ## Thanks 17 | 18 | Thank you for keeping vishwa.ai and our users safe. 🙇 -------------------------------------------------------------------------------- /__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vishwa-labs/vishwa-ml-sdk/0491d3c5a1e2301643e1fc8e0f18cfaf503b91c7/__init__.py -------------------------------------------------------------------------------- /_config.yml: -------------------------------------------------------------------------------- 1 | plugins: 2 | - jekyll-relative-links 3 | relative_links: 4 | enabled: true 5 | collections: true 6 | include: 7 | - CONTRIBUTING.md 8 | - README.md 9 | - LICENSE.md 10 | - COPYING.md 11 | - CODE_OF_CONDUCT.md 12 | - CONTRIBUTING.md 13 | - ISSUE_TEMPLATE.md 14 | - PULL_REQUEST_TEMPLATE.md 15 | -------------------------------------------------------------------------------- /dashboards/grafana_langchain.json: -------------------------------------------------------------------------------- 1 | { 2 | "__inputs": [ 3 | { 4 | "name": "DS_PROMETHEUS", 5 | "label": "Prometheus", 6 | "description": "", 7 | "type": "datasource", 8 | "pluginId": "prometheus", 9 | "pluginName": "Prometheus" 10 | } 11 | ], 12 | "__elements": {}, 13 | "__requires": [ 14 | { 15 | "type": "panel", 16 | "id": "bargauge", 17 | "name": "Bar gauge", 18 | "version": "" 19 | }, 20 | { 21 | "type": "grafana", 22 | "id": "grafana", 23 | "name": "Grafana", 24 | "version": "10.1.1" 25 | }, 26 | { 27 | "type": "datasource", 28 | "id": "prometheus", 29 | "name": "Prometheus", 30 | "version": "1.0.0" 31 | }, 32 | { 33 | "type": "panel", 34 | "id": "stat", 35 | "name": "Stat", 36 | "version": "" 37 | }, 38 | { 39 | "type": "panel", 40 | "id": "timeseries", 41 | "name": "Time series", 42 | "version": "" 43 | } 44 | ], 45 | "annotations": { 46 | "list": [ 47 | { 48 | "builtIn": 1, 49 | "datasource": { 50 | "type": "grafana", 51 | "uid": "-- Grafana --" 52 | }, 53 | "enable": true, 54 | "hide": true, 55 | "iconColor": "rgba(0, 211, 255, 1)", 56 | "name": "Annotations & Alerts", 57 | "type": "dashboard" 58 | } 59 | ] 60 | }, 61 | "description": "", 62 | "editable": true, 63 | "fiscalYearStartMonth": 0, 64 | "graphTooltip": 0, 65 | "id": null, 66 | "links": [], 67 | "liveNow": false, 68 | "panels": [ 69 | { 70 | "collapsed": false, 71 | "gridPos": { 72 | "h": 1, 73 | "w": 24, 74 | "x": 0, 75 | "y": 0 76 | }, 77 | "id": 7, 78 | "panels": [], 79 | "title": "Langchain - Agent: $agent_name", 80 | "type": "row" 81 | }, 82 | { 83 | "datasource": { 84 | "type": "prometheus", 85 | "uid": "${DS_PROMETHEUS}" 86 | }, 87 | "fieldConfig": { 88 | "defaults": { 89 | "color": { 90 | "mode": "palette-classic" 91 | }, 92 | "decimals": 0, 93 | "mappings": [], 94 | "thresholds": { 95 | "mode": "absolute", 96 | "steps": [ 97 | { 98 | "color": "green", 99 | "value": null 100 | }, 101 | { 102 | "color": "red", 103 | "value": 80 104 | } 105 | ] 106 | }, 107 | "unit": "percent" 108 | }, 109 | "overrides": [] 110 | }, 111 | "gridPos": { 112 | "h": 8, 113 | "w": 12, 114 | "x": 0, 115 | "y": 1 116 | }, 117 | "id": 8, 118 | "options": { 119 | "displayMode": "gradient", 120 | "minVizHeight": 10, 121 | "minVizWidth": 0, 122 | "orientation": "auto", 123 | "reduceOptions": { 124 | "calcs": [ 125 | "lastNotNull" 126 | ], 127 | "fields": "", 128 | "values": false 129 | }, 130 | "showUnfilled": true, 131 | "valueMode": "color" 132 | }, 133 | "pluginVersion": "10.1.1", 134 | "targets": [ 135 | { 136 | "datasource": { 137 | "type": "prometheus", 138 | "uid": "${DS_PROMETHEUS}" 139 | }, 140 | "editorMode": "code", 141 | "exemplar": false, 142 | "expr": "sort(\n 100 * \n increase(langchain_agent_run_latency_bucket{agent_name=\"$agent_name\"}[1h]) \n / \n scalar(sum(increase(langchain_agent_run_latency_bucket{agent_name=\"$agent_name\"}[1h])))\n)\n", 143 | "hide": false, 144 | "instant": true, 145 | "legendFormat": "{{le}}s", 146 | "range": false, 147 | "refId": "B" 148 | } 149 | ], 150 | "title": "Langchain - Chain Latency Distribution Past 1h", 151 | "type": "bargauge" 152 | }, 153 | { 154 | "datasource": { 155 | "type": "prometheus", 156 | "uid": "${DS_PROMETHEUS}" 157 | }, 158 | "fieldConfig": { 159 | "defaults": { 160 | "color": { 161 | "mode": "palette-classic" 162 | }, 163 | "custom": { 164 | "axisCenteredZero": false, 165 | "axisColorMode": "text", 166 | "axisLabel": "", 167 | "axisPlacement": "auto", 168 | "barAlignment": 0, 169 | "drawStyle": "line", 170 | "fillOpacity": 0, 171 | "gradientMode": "none", 172 | "hideFrom": { 173 | "legend": false, 174 | "tooltip": false, 175 | "viz": false 176 | }, 177 | "insertNulls": false, 178 | "lineInterpolation": "linear", 179 | "lineWidth": 1, 180 | "pointSize": 5, 181 | "scaleDistribution": { 182 | "type": "linear" 183 | }, 184 | "showPoints": "auto", 185 | "spanNulls": false, 186 | "stacking": { 187 | "group": "A", 188 | "mode": "none" 189 | }, 190 | "thresholdsStyle": { 191 | "mode": "off" 192 | } 193 | }, 194 | "decimals": 0, 195 | "mappings": [], 196 | "thresholds": { 197 | "mode": "absolute", 198 | "steps": [ 199 | { 200 | "color": "green", 201 | "value": null 202 | }, 203 | { 204 | "color": "red", 205 | "value": 80 206 | } 207 | ] 208 | }, 209 | "unit": "s" 210 | }, 211 | "overrides": [] 212 | }, 213 | "gridPos": { 214 | "h": 8, 215 | "w": 12, 216 | "x": 12, 217 | "y": 1 218 | }, 219 | "id": 9, 220 | "options": { 221 | "legend": { 222 | "calcs": [], 223 | "displayMode": "list", 224 | "placement": "bottom", 225 | "showLegend": true 226 | }, 227 | "tooltip": { 228 | "mode": "single", 229 | "sort": "none" 230 | } 231 | }, 232 | "targets": [ 233 | { 234 | "datasource": { 235 | "type": "prometheus", 236 | "uid": "${DS_PROMETHEUS}" 237 | }, 238 | "disableTextWrap": false, 239 | "editorMode": "code", 240 | "expr": "sum(rate(langchain_agent_run_latency_sum{agent_name=\"$agent_name\"}[1m])) / sum(rate(langchain_agent_run_latency_count{agent_name=\"$agent_name\"}[1m]))\n", 241 | "fullMetaSearch": false, 242 | "includeNullMetadata": true, 243 | "instant": false, 244 | "legendFormat": "Average Latency", 245 | "range": true, 246 | "refId": "A", 247 | "useBackend": false 248 | }, 249 | { 250 | "datasource": { 251 | "type": "prometheus", 252 | "uid": "${DS_PROMETHEUS}" 253 | }, 254 | "editorMode": "code", 255 | "expr": "histogram_quantile(0.90, sum(rate(langchain_agent_run_latency_bucket{agent_name=\"$agent_name\"}[1m])) by (le))\n", 256 | "hide": false, 257 | "instant": false, 258 | "legendFormat": "p90", 259 | "range": true, 260 | "refId": "B" 261 | }, 262 | { 263 | "datasource": { 264 | "type": "prometheus", 265 | "uid": "${DS_PROMETHEUS}" 266 | }, 267 | "editorMode": "code", 268 | "expr": "histogram_quantile(0.95, sum(rate(langchain_agent_run_latency_bucket{agent_name=\"$agent_name\"}[1m])) by (le))\n", 269 | "hide": false, 270 | "instant": false, 271 | "legendFormat": "p95", 272 | "range": true, 273 | "refId": "C" 274 | }, 275 | { 276 | "datasource": { 277 | "type": "prometheus", 278 | "uid": "${DS_PROMETHEUS}" 279 | }, 280 | "editorMode": "code", 281 | "expr": "histogram_quantile(0.99, sum(rate(langchain_agent_run_latency_bucket[1m])) by (le))\n", 282 | "hide": false, 283 | "instant": false, 284 | "legendFormat": "p99", 285 | "range": true, 286 | "refId": "D" 287 | } 288 | ], 289 | "title": "Langchain - Agent Latency", 290 | "type": "timeseries" 291 | }, 292 | { 293 | "collapsed": false, 294 | "gridPos": { 295 | "h": 1, 296 | "w": 24, 297 | "x": 0, 298 | "y": 9 299 | }, 300 | "id": 6, 301 | "panels": [], 302 | "title": "Langchain - Chains", 303 | "type": "row" 304 | }, 305 | { 306 | "datasource": { 307 | "type": "prometheus", 308 | "uid": "${DS_PROMETHEUS}" 309 | }, 310 | "fieldConfig": { 311 | "defaults": { 312 | "color": { 313 | "mode": "palette-classic" 314 | }, 315 | "custom": { 316 | "axisCenteredZero": false, 317 | "axisColorMode": "text", 318 | "axisLabel": "", 319 | "axisPlacement": "auto", 320 | "barAlignment": 0, 321 | "drawStyle": "line", 322 | "fillOpacity": 0, 323 | "gradientMode": "none", 324 | "hideFrom": { 325 | "legend": false, 326 | "tooltip": false, 327 | "viz": false 328 | }, 329 | "insertNulls": false, 330 | "lineInterpolation": "linear", 331 | "lineWidth": 1, 332 | "pointSize": 5, 333 | "scaleDistribution": { 334 | "type": "linear" 335 | }, 336 | "showPoints": "auto", 337 | "spanNulls": false, 338 | "stacking": { 339 | "group": "A", 340 | "mode": "none" 341 | }, 342 | "thresholdsStyle": { 343 | "mode": "off" 344 | } 345 | }, 346 | "mappings": [], 347 | "thresholds": { 348 | "mode": "absolute", 349 | "steps": [ 350 | { 351 | "color": "green", 352 | "value": null 353 | }, 354 | { 355 | "color": "red", 356 | "value": 80 357 | } 358 | ] 359 | }, 360 | "unit": "none" 361 | }, 362 | "overrides": [] 363 | }, 364 | "gridPos": { 365 | "h": 8, 366 | "w": 12, 367 | "x": 0, 368 | "y": 10 369 | }, 370 | "id": 1, 371 | "options": { 372 | "legend": { 373 | "calcs": [], 374 | "displayMode": "list", 375 | "placement": "bottom", 376 | "showLegend": true 377 | }, 378 | "tooltip": { 379 | "mode": "single", 380 | "sort": "none" 381 | } 382 | }, 383 | "targets": [ 384 | { 385 | "datasource": { 386 | "type": "prometheus", 387 | "uid": "${DS_PROMETHEUS}" 388 | }, 389 | "disableTextWrap": false, 390 | "editorMode": "code", 391 | "expr": "rate(langchain_chain_execution_total{execution_step=\"on_chain_start\", agent_name=\"$agent_name\"}[1m])", 392 | "fullMetaSearch": false, 393 | "includeNullMetadata": true, 394 | "instant": false, 395 | "legendFormat": "{{ml_model_name}},{{ml_model_type}}", 396 | "range": true, 397 | "refId": "A", 398 | "useBackend": false 399 | } 400 | ], 401 | "title": "Chain Executions Per Minute", 402 | "type": "timeseries" 403 | }, 404 | { 405 | "datasource": { 406 | "type": "prometheus", 407 | "uid": "${DS_PROMETHEUS}" 408 | }, 409 | "fieldConfig": { 410 | "defaults": { 411 | "color": { 412 | "mode": "palette-classic" 413 | }, 414 | "custom": { 415 | "axisCenteredZero": false, 416 | "axisColorMode": "text", 417 | "axisLabel": "", 418 | "axisPlacement": "auto", 419 | "barAlignment": 0, 420 | "drawStyle": "line", 421 | "fillOpacity": 0, 422 | "gradientMode": "none", 423 | "hideFrom": { 424 | "legend": false, 425 | "tooltip": false, 426 | "viz": false 427 | }, 428 | "insertNulls": false, 429 | "lineInterpolation": "linear", 430 | "lineWidth": 1, 431 | "pointSize": 5, 432 | "scaleDistribution": { 433 | "type": "linear" 434 | }, 435 | "showPoints": "auto", 436 | "spanNulls": false, 437 | "stacking": { 438 | "group": "A", 439 | "mode": "none" 440 | }, 441 | "thresholdsStyle": { 442 | "mode": "off" 443 | } 444 | }, 445 | "decimals": 0, 446 | "mappings": [], 447 | "thresholds": { 448 | "mode": "absolute", 449 | "steps": [ 450 | { 451 | "color": "green", 452 | "value": null 453 | }, 454 | { 455 | "color": "red", 456 | "value": 80 457 | } 458 | ] 459 | }, 460 | "unit": "s" 461 | }, 462 | "overrides": [] 463 | }, 464 | "gridPos": { 465 | "h": 8, 466 | "w": 12, 467 | "x": 12, 468 | "y": 10 469 | }, 470 | "id": 3, 471 | "options": { 472 | "legend": { 473 | "calcs": [], 474 | "displayMode": "list", 475 | "placement": "bottom", 476 | "showLegend": true 477 | }, 478 | "tooltip": { 479 | "mode": "single", 480 | "sort": "none" 481 | } 482 | }, 483 | "targets": [ 484 | { 485 | "datasource": { 486 | "type": "prometheus", 487 | "uid": "${DS_PROMETHEUS}" 488 | }, 489 | "disableTextWrap": false, 490 | "editorMode": "code", 491 | "expr": "sum(rate(langchain_chain_latency_sum{ml_model_name=\"$model_name\", ml_model_type=\"$model_type\", agent_name=\"$agent_name\"}[1m])) / sum(rate(langchain_chain_latency_count{ml_model_name=\"$model_name\", ml_model_type=\"$model_type\", agent_name=\"$agent_name\"}[1m]))\n", 492 | "fullMetaSearch": false, 493 | "includeNullMetadata": true, 494 | "instant": false, 495 | "legendFormat": "Average Latency", 496 | "range": true, 497 | "refId": "A", 498 | "useBackend": false 499 | }, 500 | { 501 | "datasource": { 502 | "type": "prometheus", 503 | "uid": "${DS_PROMETHEUS}" 504 | }, 505 | "editorMode": "code", 506 | "expr": "histogram_quantile(0.90, sum(rate(langchain_chain_latency_bucket{ml_model_name=\"$model_name\", ml_model_type=\"$model_type\", agent_name=\"$agent_name\"}[1m])) by (le))\n", 507 | "hide": false, 508 | "instant": false, 509 | "legendFormat": "p90", 510 | "range": true, 511 | "refId": "B" 512 | }, 513 | { 514 | "datasource": { 515 | "type": "prometheus", 516 | "uid": "${DS_PROMETHEUS}" 517 | }, 518 | "editorMode": "code", 519 | "expr": "histogram_quantile(0.95, sum(rate(langchain_chain_latency_bucket{ml_model_name=\"$model_name\", ml_model_type=\"$model_type\", agent_name=\"$agent_name\"}[1m])) by (le))\n", 520 | "hide": false, 521 | "instant": false, 522 | "legendFormat": "p95", 523 | "range": true, 524 | "refId": "C" 525 | }, 526 | { 527 | "datasource": { 528 | "type": "prometheus", 529 | "uid": "${DS_PROMETHEUS}" 530 | }, 531 | "editorMode": "code", 532 | "expr": "histogram_quantile(0.99, sum(rate(langchain_chain_latency_bucket{ml_model_name=\"$model_name\", ml_model_type=\"$model_type\", agent_name=\"$agent_name\"}[1m])) by (le))\n", 533 | "hide": false, 534 | "instant": false, 535 | "legendFormat": "p99", 536 | "range": true, 537 | "refId": "D" 538 | } 539 | ], 540 | "title": "Langchain - Chain Latency", 541 | "type": "timeseries" 542 | }, 543 | { 544 | "datasource": { 545 | "type": "prometheus", 546 | "uid": "${DS_PROMETHEUS}" 547 | }, 548 | "fieldConfig": { 549 | "defaults": { 550 | "color": { 551 | "mode": "palette-classic" 552 | }, 553 | "decimals": 0, 554 | "mappings": [], 555 | "thresholds": { 556 | "mode": "absolute", 557 | "steps": [ 558 | { 559 | "color": "green", 560 | "value": null 561 | }, 562 | { 563 | "color": "red", 564 | "value": 80 565 | } 566 | ] 567 | }, 568 | "unit": "percent" 569 | }, 570 | "overrides": [] 571 | }, 572 | "gridPos": { 573 | "h": 8, 574 | "w": 12, 575 | "x": 0, 576 | "y": 18 577 | }, 578 | "id": 4, 579 | "options": { 580 | "displayMode": "gradient", 581 | "minVizHeight": 10, 582 | "minVizWidth": 0, 583 | "orientation": "auto", 584 | "reduceOptions": { 585 | "calcs": [ 586 | "lastNotNull" 587 | ], 588 | "fields": "", 589 | "values": false 590 | }, 591 | "showUnfilled": true, 592 | "valueMode": "color" 593 | }, 594 | "pluginVersion": "10.1.1", 595 | "targets": [ 596 | { 597 | "datasource": { 598 | "type": "prometheus", 599 | "uid": "${DS_PROMETHEUS}" 600 | }, 601 | "editorMode": "code", 602 | "exemplar": false, 603 | "expr": "sort(sum(increase(langchain_chain_latency_bucket{ml_model_type=\"$model_type\", ml_model_name=\"$model_name\", agent_name=\"$agent_name\"}[3h])) by (le))", 604 | "format": "time_series", 605 | "hide": true, 606 | "instant": true, 607 | "legendFormat": "{{le}}", 608 | "range": false, 609 | "refId": "A" 610 | }, 611 | { 612 | "datasource": { 613 | "type": "prometheus", 614 | "uid": "${DS_PROMETHEUS}" 615 | }, 616 | "editorMode": "code", 617 | "exemplar": false, 618 | "expr": "sort(\n 100 * \n increase(langchain_chain_latency_bucket{ml_model_type=\"$model_type\", ml_model_name=\"$model_name\", agent_name=\"$agent_name\"}[3h]) \n / \n scalar(sum(increase(langchain_chain_latency_bucket{ml_model_type=\"$model_type\", ml_model_name=\"$model_name\", agent_name=\"$agent_name\"}[3h])))\n)\n", 619 | "hide": false, 620 | "instant": true, 621 | "legendFormat": "{{le}}s", 622 | "range": false, 623 | "refId": "B" 624 | } 625 | ], 626 | "title": "Langchain - Chain Latency Distribution Past 3h", 627 | "type": "bargauge" 628 | }, 629 | { 630 | "datasource": { 631 | "type": "prometheus", 632 | "uid": "${DS_PROMETHEUS}" 633 | }, 634 | "fieldConfig": { 635 | "defaults": { 636 | "color": { 637 | "mode": "palette-classic" 638 | }, 639 | "decimals": 0, 640 | "mappings": [], 641 | "thresholds": { 642 | "mode": "absolute", 643 | "steps": [ 644 | { 645 | "color": "green", 646 | "value": null 647 | }, 648 | { 649 | "color": "red", 650 | "value": 80 651 | } 652 | ] 653 | }, 654 | "unit": "percent" 655 | }, 656 | "overrides": [] 657 | }, 658 | "gridPos": { 659 | "h": 8, 660 | "w": 12, 661 | "x": 12, 662 | "y": 18 663 | }, 664 | "id": 5, 665 | "options": { 666 | "displayMode": "gradient", 667 | "minVizHeight": 10, 668 | "minVizWidth": 0, 669 | "orientation": "auto", 670 | "reduceOptions": { 671 | "calcs": [ 672 | "lastNotNull" 673 | ], 674 | "fields": "", 675 | "values": false 676 | }, 677 | "showUnfilled": true, 678 | "valueMode": "color" 679 | }, 680 | "pluginVersion": "10.1.1", 681 | "targets": [ 682 | { 683 | "datasource": { 684 | "type": "prometheus", 685 | "uid": "${DS_PROMETHEUS}" 686 | }, 687 | "editorMode": "code", 688 | "exemplar": false, 689 | "expr": "sort(\n 100 * \n increase(langchain_chain_latency_bucket{ml_model_type=\"$model_type\", ml_model_name=\"$model_name\", agent_name=\"$agent_name\"}[1h]) \n / \n scalar(sum(increase(langchain_chain_latency_bucket{ml_model_type=\"$model_type\", ml_model_name=\"$model_name\", agent_name=\"$agent_name\"}[1h])))\n)\n", 690 | "hide": false, 691 | "instant": true, 692 | "legendFormat": "{{le}}s", 693 | "range": false, 694 | "refId": "B" 695 | } 696 | ], 697 | "title": "Langchain - Chain Latency Distribution Past 1h", 698 | "type": "bargauge" 699 | }, 700 | { 701 | "datasource": { 702 | "type": "prometheus", 703 | "uid": "${DS_PROMETHEUS}" 704 | }, 705 | "fieldConfig": { 706 | "defaults": { 707 | "color": { 708 | "mode": "thresholds" 709 | }, 710 | "mappings": [], 711 | "thresholds": { 712 | "mode": "absolute", 713 | "steps": [ 714 | { 715 | "color": "green" 716 | }, 717 | { 718 | "color": "red", 719 | "value": 80 720 | } 721 | ] 722 | }, 723 | "unit": "short" 724 | }, 725 | "overrides": [] 726 | }, 727 | "gridPos": { 728 | "h": 4, 729 | "w": 6, 730 | "x": 0, 731 | "y": 26 732 | }, 733 | "id": 26, 734 | "options": { 735 | "colorMode": "value", 736 | "graphMode": "area", 737 | "justifyMode": "auto", 738 | "orientation": "auto", 739 | "reduceOptions": { 740 | "calcs": [ 741 | "lastNotNull" 742 | ], 743 | "fields": "", 744 | "values": false 745 | }, 746 | "textMode": "auto" 747 | }, 748 | "pluginVersion": "10.1.1", 749 | "targets": [ 750 | { 751 | "datasource": { 752 | "type": "prometheus", 753 | "uid": "${DS_PROMETHEUS}" 754 | }, 755 | "disableTextWrap": false, 756 | "editorMode": "code", 757 | "expr": "increase(langchain_chain_execution_total{execution_step=\"on_chain_end\", agent_name=\"$agent_name\", ml_model_type=\"$model_type\", ml_model_name=\"$model_name\"}[1h])\n", 758 | "fullMetaSearch": false, 759 | "includeNullMetadata": true, 760 | "instant": false, 761 | "legendFormat": "{{usage_type}}", 762 | "range": true, 763 | "refId": "A", 764 | "useBackend": false 765 | } 766 | ], 767 | "title": "Langchain Chain Usage past 1hr - Success", 768 | "type": "stat" 769 | }, 770 | { 771 | "datasource": { 772 | "type": "prometheus", 773 | "uid": "${DS_PROMETHEUS}" 774 | }, 775 | "fieldConfig": { 776 | "defaults": { 777 | "color": { 778 | "mode": "thresholds" 779 | }, 780 | "mappings": [], 781 | "thresholds": { 782 | "mode": "absolute", 783 | "steps": [ 784 | { 785 | "color": "green" 786 | }, 787 | { 788 | "color": "red", 789 | "value": 80 790 | } 791 | ] 792 | }, 793 | "unit": "short" 794 | }, 795 | "overrides": [] 796 | }, 797 | "gridPos": { 798 | "h": 4, 799 | "w": 6, 800 | "x": 6, 801 | "y": 26 802 | }, 803 | "id": 27, 804 | "options": { 805 | "colorMode": "value", 806 | "graphMode": "area", 807 | "justifyMode": "auto", 808 | "orientation": "auto", 809 | "reduceOptions": { 810 | "calcs": [ 811 | "lastNotNull" 812 | ], 813 | "fields": "", 814 | "values": false 815 | }, 816 | "textMode": "auto" 817 | }, 818 | "pluginVersion": "10.1.1", 819 | "targets": [ 820 | { 821 | "datasource": { 822 | "type": "prometheus", 823 | "uid": "${DS_PROMETHEUS}" 824 | }, 825 | "disableTextWrap": false, 826 | "editorMode": "code", 827 | "expr": "increase(langchain_chain_execution_total{execution_step=\"on_chain_error\", agent_name=\"$agent_name\", ml_model_type=\"$model_type\", ml_model_name=\"$model_name\"}[1h])\n", 828 | "fullMetaSearch": false, 829 | "includeNullMetadata": true, 830 | "instant": false, 831 | "legendFormat": "{{usage_type}}", 832 | "range": true, 833 | "refId": "A", 834 | "useBackend": false 835 | } 836 | ], 837 | "title": "Langchain Chain Usage past 1hr - Error", 838 | "type": "stat" 839 | }, 840 | { 841 | "collapsed": false, 842 | "gridPos": { 843 | "h": 1, 844 | "w": 24, 845 | "x": 0, 846 | "y": 30 847 | }, 848 | "id": 10, 849 | "panels": [], 850 | "title": "Langchain - LLM Usage", 851 | "type": "row" 852 | }, 853 | { 854 | "datasource": { 855 | "type": "prometheus", 856 | "uid": "${DS_PROMETHEUS}" 857 | }, 858 | "fieldConfig": { 859 | "defaults": { 860 | "color": { 861 | "mode": "palette-classic" 862 | }, 863 | "custom": { 864 | "axisCenteredZero": false, 865 | "axisColorMode": "text", 866 | "axisLabel": "", 867 | "axisPlacement": "auto", 868 | "barAlignment": 0, 869 | "drawStyle": "line", 870 | "fillOpacity": 0, 871 | "gradientMode": "none", 872 | "hideFrom": { 873 | "legend": false, 874 | "tooltip": false, 875 | "viz": false 876 | }, 877 | "insertNulls": false, 878 | "lineInterpolation": "linear", 879 | "lineWidth": 1, 880 | "pointSize": 5, 881 | "scaleDistribution": { 882 | "type": "linear" 883 | }, 884 | "showPoints": "auto", 885 | "spanNulls": false, 886 | "stacking": { 887 | "group": "A", 888 | "mode": "none" 889 | }, 890 | "thresholdsStyle": { 891 | "mode": "off" 892 | } 893 | }, 894 | "mappings": [], 895 | "thresholds": { 896 | "mode": "absolute", 897 | "steps": [ 898 | { 899 | "color": "green" 900 | }, 901 | { 902 | "color": "red", 903 | "value": 80 904 | } 905 | ] 906 | }, 907 | "unit": "short" 908 | }, 909 | "overrides": [] 910 | }, 911 | "gridPos": { 912 | "h": 8, 913 | "w": 12, 914 | "x": 0, 915 | "y": 31 916 | }, 917 | "id": 2, 918 | "options": { 919 | "legend": { 920 | "calcs": [], 921 | "displayMode": "list", 922 | "placement": "bottom", 923 | "showLegend": true 924 | }, 925 | "tooltip": { 926 | "mode": "single", 927 | "sort": "none" 928 | } 929 | }, 930 | "targets": [ 931 | { 932 | "datasource": { 933 | "type": "prometheus", 934 | "uid": "${DS_PROMETHEUS}" 935 | }, 936 | "disableTextWrap": false, 937 | "editorMode": "code", 938 | "expr": "rate(langchain_chat_model_total{execution_step=\"on_llm_end\", agent_name=\"$agent_name\"}[1m])", 939 | "fullMetaSearch": false, 940 | "includeNullMetadata": true, 941 | "instant": false, 942 | "legendFormat": "{{ml_model_name}},{{ml_model_type}}", 943 | "range": true, 944 | "refId": "A", 945 | "useBackend": false 946 | } 947 | ], 948 | "title": "LLM Executions Per Minute", 949 | "type": "timeseries" 950 | }, 951 | { 952 | "datasource": { 953 | "type": "prometheus", 954 | "uid": "${DS_PROMETHEUS}" 955 | }, 956 | "fieldConfig": { 957 | "defaults": { 958 | "color": { 959 | "mode": "palette-classic" 960 | }, 961 | "custom": { 962 | "axisCenteredZero": false, 963 | "axisColorMode": "text", 964 | "axisLabel": "", 965 | "axisPlacement": "auto", 966 | "barAlignment": 0, 967 | "drawStyle": "line", 968 | "fillOpacity": 0, 969 | "gradientMode": "none", 970 | "hideFrom": { 971 | "legend": false, 972 | "tooltip": false, 973 | "viz": false 974 | }, 975 | "insertNulls": false, 976 | "lineInterpolation": "linear", 977 | "lineWidth": 1, 978 | "pointSize": 5, 979 | "scaleDistribution": { 980 | "type": "linear" 981 | }, 982 | "showPoints": "auto", 983 | "spanNulls": false, 984 | "stacking": { 985 | "group": "A", 986 | "mode": "none" 987 | }, 988 | "thresholdsStyle": { 989 | "mode": "off" 990 | } 991 | }, 992 | "decimals": 0, 993 | "mappings": [], 994 | "thresholds": { 995 | "mode": "absolute", 996 | "steps": [ 997 | { 998 | "color": "green" 999 | }, 1000 | { 1001 | "color": "red", 1002 | "value": 80 1003 | } 1004 | ] 1005 | }, 1006 | "unit": "s" 1007 | }, 1008 | "overrides": [] 1009 | }, 1010 | "gridPos": { 1011 | "h": 8, 1012 | "w": 12, 1013 | "x": 12, 1014 | "y": 31 1015 | }, 1016 | "id": 11, 1017 | "options": { 1018 | "legend": { 1019 | "calcs": [], 1020 | "displayMode": "list", 1021 | "placement": "bottom", 1022 | "showLegend": true 1023 | }, 1024 | "tooltip": { 1025 | "mode": "single", 1026 | "sort": "none" 1027 | } 1028 | }, 1029 | "targets": [ 1030 | { 1031 | "datasource": { 1032 | "type": "prometheus", 1033 | "uid": "${DS_PROMETHEUS}" 1034 | }, 1035 | "disableTextWrap": false, 1036 | "editorMode": "code", 1037 | "expr": "sum(rate(langchain_chat_model_latency_sum{ml_model_name=\"$model_name\", ml_model_type=\"$model_type\", agent_name=\"$agent_name\"}[1m])) / sum(rate(langchain_chat_model_latency_count{ml_model_name=\"$model_name\", ml_model_type=\"$model_type\", agent_name=\"$agent_name\"}[1m]))\n", 1038 | "fullMetaSearch": false, 1039 | "includeNullMetadata": true, 1040 | "instant": false, 1041 | "legendFormat": "Average Latency", 1042 | "range": true, 1043 | "refId": "A", 1044 | "useBackend": false 1045 | }, 1046 | { 1047 | "datasource": { 1048 | "type": "prometheus", 1049 | "uid": "${DS_PROMETHEUS}" 1050 | }, 1051 | "editorMode": "code", 1052 | "expr": "histogram_quantile(0.90, sum(rate(langchain_chat_model_latency_bucket{ml_model_name=\"$model_name\", ml_model_type=\"$model_type\", agent_name=\"$agent_name\"}[1m])) by (le))\n", 1053 | "hide": false, 1054 | "instant": false, 1055 | "legendFormat": "p90", 1056 | "range": true, 1057 | "refId": "B" 1058 | }, 1059 | { 1060 | "datasource": { 1061 | "type": "prometheus", 1062 | "uid": "${DS_PROMETHEUS}" 1063 | }, 1064 | "editorMode": "code", 1065 | "expr": "histogram_quantile(0.95, sum(rate(langchain_chat_model_latency_bucket{ml_model_name=\"$model_name\", ml_model_type=\"$model_type\", agent_name=\"$agent_name\"}[1m])) by (le))\n", 1066 | "hide": false, 1067 | "instant": false, 1068 | "legendFormat": "p95", 1069 | "range": true, 1070 | "refId": "C" 1071 | }, 1072 | { 1073 | "datasource": { 1074 | "type": "prometheus", 1075 | "uid": "${DS_PROMETHEUS}" 1076 | }, 1077 | "editorMode": "code", 1078 | "expr": "histogram_quantile(0.99, sum(rate(langchain_chat_model_latency_bucket{ml_model_name=\"$model_name\", ml_model_type=\"$model_type\", agent_name=\"$agent_name\"}[1m])) by (le))\n", 1079 | "hide": false, 1080 | "instant": false, 1081 | "legendFormat": "p99", 1082 | "range": true, 1083 | "refId": "D" 1084 | } 1085 | ], 1086 | "title": "Langchain - LLM Latency", 1087 | "type": "timeseries" 1088 | }, 1089 | { 1090 | "datasource": { 1091 | "type": "prometheus", 1092 | "uid": "${DS_PROMETHEUS}" 1093 | }, 1094 | "fieldConfig": { 1095 | "defaults": { 1096 | "color": { 1097 | "mode": "palette-classic" 1098 | }, 1099 | "decimals": 0, 1100 | "mappings": [], 1101 | "thresholds": { 1102 | "mode": "absolute", 1103 | "steps": [ 1104 | { 1105 | "color": "green" 1106 | }, 1107 | { 1108 | "color": "red", 1109 | "value": 80 1110 | } 1111 | ] 1112 | }, 1113 | "unit": "percent" 1114 | }, 1115 | "overrides": [] 1116 | }, 1117 | "gridPos": { 1118 | "h": 8, 1119 | "w": 12, 1120 | "x": 0, 1121 | "y": 39 1122 | }, 1123 | "id": 12, 1124 | "options": { 1125 | "displayMode": "gradient", 1126 | "minVizHeight": 10, 1127 | "minVizWidth": 0, 1128 | "orientation": "auto", 1129 | "reduceOptions": { 1130 | "calcs": [ 1131 | "lastNotNull" 1132 | ], 1133 | "fields": "", 1134 | "values": false 1135 | }, 1136 | "showUnfilled": true, 1137 | "valueMode": "color" 1138 | }, 1139 | "pluginVersion": "10.1.1", 1140 | "targets": [ 1141 | { 1142 | "datasource": { 1143 | "type": "prometheus", 1144 | "uid": "${DS_PROMETHEUS}" 1145 | }, 1146 | "editorMode": "code", 1147 | "exemplar": false, 1148 | "expr": "sort(sum(increase(langchain_chat_model_latency_bucket{ml_model_type=\"$model_type\", ml_model_name=\"$model_name\", agent_name=\"$agent_name\"}[3h])) by (le))", 1149 | "format": "time_series", 1150 | "hide": true, 1151 | "instant": true, 1152 | "legendFormat": "{{le}}", 1153 | "range": false, 1154 | "refId": "A" 1155 | }, 1156 | { 1157 | "datasource": { 1158 | "type": "prometheus", 1159 | "uid": "${DS_PROMETHEUS}" 1160 | }, 1161 | "editorMode": "code", 1162 | "exemplar": false, 1163 | "expr": "sort(\n 100 * \n increase(langchain_chat_model_latency_bucket{ml_model_type=\"$model_type\", ml_model_name=\"$model_name\", agent_name=\"$agent_name\"}[3h]) \n / \n scalar(sum(increase(langchain_chat_model_latency_bucket{ml_model_type=\"$model_type\", ml_model_name=\"$model_name\", agent_name=\"$agent_name\"}[3h])))\n)\n", 1164 | "hide": false, 1165 | "instant": true, 1166 | "legendFormat": "{{le}}s", 1167 | "range": false, 1168 | "refId": "B" 1169 | } 1170 | ], 1171 | "title": "Langchain - LLM Latency Distribution Past 3h", 1172 | "type": "bargauge" 1173 | }, 1174 | { 1175 | "datasource": { 1176 | "type": "prometheus", 1177 | "uid": "${DS_PROMETHEUS}" 1178 | }, 1179 | "fieldConfig": { 1180 | "defaults": { 1181 | "color": { 1182 | "mode": "palette-classic" 1183 | }, 1184 | "decimals": 0, 1185 | "mappings": [], 1186 | "thresholds": { 1187 | "mode": "absolute", 1188 | "steps": [ 1189 | { 1190 | "color": "green" 1191 | }, 1192 | { 1193 | "color": "red", 1194 | "value": 80 1195 | } 1196 | ] 1197 | }, 1198 | "unit": "percent" 1199 | }, 1200 | "overrides": [] 1201 | }, 1202 | "gridPos": { 1203 | "h": 8, 1204 | "w": 12, 1205 | "x": 12, 1206 | "y": 39 1207 | }, 1208 | "id": 22, 1209 | "options": { 1210 | "displayMode": "gradient", 1211 | "minVizHeight": 10, 1212 | "minVizWidth": 0, 1213 | "orientation": "auto", 1214 | "reduceOptions": { 1215 | "calcs": [ 1216 | "lastNotNull" 1217 | ], 1218 | "fields": "", 1219 | "values": false 1220 | }, 1221 | "showUnfilled": true, 1222 | "valueMode": "color" 1223 | }, 1224 | "pluginVersion": "10.1.1", 1225 | "targets": [ 1226 | { 1227 | "datasource": { 1228 | "type": "prometheus", 1229 | "uid": "${DS_PROMETHEUS}" 1230 | }, 1231 | "editorMode": "code", 1232 | "exemplar": false, 1233 | "expr": "sort(sum(increase(langchain_chat_model_latency_bucket{ml_model_type=\"$model_type\", ml_model_name=\"$model_name\", agent_name=\"$agent_name\"}[3h])) by (le))", 1234 | "format": "time_series", 1235 | "hide": true, 1236 | "instant": true, 1237 | "legendFormat": "{{le}}", 1238 | "range": false, 1239 | "refId": "A" 1240 | }, 1241 | { 1242 | "datasource": { 1243 | "type": "prometheus", 1244 | "uid": "${DS_PROMETHEUS}" 1245 | }, 1246 | "editorMode": "code", 1247 | "exemplar": false, 1248 | "expr": "sort(\n 100 * \n increase(langchain_chat_model_latency_bucket{ml_model_type=\"$model_type\", ml_model_name=\"$model_name\", agent_name=\"$agent_name\"}[3h]) \n / \n scalar(sum(increase(langchain_chat_model_latency_bucket{ml_model_type=\"$model_type\", ml_model_name=\"$model_name\", agent_name=\"$agent_name\"}[3h])))\n)\n", 1249 | "hide": false, 1250 | "instant": true, 1251 | "legendFormat": "{{le}}s", 1252 | "range": false, 1253 | "refId": "B" 1254 | } 1255 | ], 1256 | "title": "Langchain - LLM Latency Distribution Past 3h", 1257 | "type": "bargauge" 1258 | }, 1259 | { 1260 | "collapsed": false, 1261 | "gridPos": { 1262 | "h": 1, 1263 | "w": 24, 1264 | "x": 0, 1265 | "y": 47 1266 | }, 1267 | "id": 14, 1268 | "panels": [], 1269 | "title": "OpenAI - Tokens Usage", 1270 | "type": "row" 1271 | }, 1272 | { 1273 | "datasource": { 1274 | "type": "prometheus", 1275 | "uid": "${DS_PROMETHEUS}" 1276 | }, 1277 | "fieldConfig": { 1278 | "defaults": { 1279 | "color": { 1280 | "mode": "palette-classic" 1281 | }, 1282 | "custom": { 1283 | "axisCenteredZero": false, 1284 | "axisColorMode": "text", 1285 | "axisLabel": "", 1286 | "axisPlacement": "auto", 1287 | "barAlignment": 0, 1288 | "drawStyle": "line", 1289 | "fillOpacity": 0, 1290 | "gradientMode": "none", 1291 | "hideFrom": { 1292 | "legend": false, 1293 | "tooltip": false, 1294 | "viz": false 1295 | }, 1296 | "insertNulls": false, 1297 | "lineInterpolation": "linear", 1298 | "lineWidth": 1, 1299 | "pointSize": 5, 1300 | "scaleDistribution": { 1301 | "type": "linear" 1302 | }, 1303 | "showPoints": "auto", 1304 | "spanNulls": false, 1305 | "stacking": { 1306 | "group": "A", 1307 | "mode": "none" 1308 | }, 1309 | "thresholdsStyle": { 1310 | "mode": "off" 1311 | } 1312 | }, 1313 | "mappings": [], 1314 | "thresholds": { 1315 | "mode": "absolute", 1316 | "steps": [ 1317 | { 1318 | "color": "green" 1319 | }, 1320 | { 1321 | "color": "red", 1322 | "value": 80 1323 | } 1324 | ] 1325 | }, 1326 | "unit": "short" 1327 | }, 1328 | "overrides": [] 1329 | }, 1330 | "gridPos": { 1331 | "h": 8, 1332 | "w": 12, 1333 | "x": 0, 1334 | "y": 48 1335 | }, 1336 | "id": 15, 1337 | "options": { 1338 | "legend": { 1339 | "calcs": [], 1340 | "displayMode": "list", 1341 | "placement": "bottom", 1342 | "showLegend": true 1343 | }, 1344 | "tooltip": { 1345 | "mode": "single", 1346 | "sort": "none" 1347 | } 1348 | }, 1349 | "targets": [ 1350 | { 1351 | "datasource": { 1352 | "type": "prometheus", 1353 | "uid": "${DS_PROMETHEUS}" 1354 | }, 1355 | "disableTextWrap": false, 1356 | "editorMode": "code", 1357 | "expr": "increase(langchain_llm_tokens_total{agent_name=\"$agent_name\", ml_model_name=\"$model_name\"}[1m])", 1358 | "fullMetaSearch": false, 1359 | "includeNullMetadata": true, 1360 | "instant": false, 1361 | "legendFormat": "{{usage_type}}", 1362 | "range": true, 1363 | "refId": "A", 1364 | "useBackend": false 1365 | } 1366 | ], 1367 | "title": "OpenAI Tokens Used per Minute", 1368 | "type": "timeseries" 1369 | }, 1370 | { 1371 | "datasource": { 1372 | "type": "prometheus", 1373 | "uid": "${DS_PROMETHEUS}" 1374 | }, 1375 | "fieldConfig": { 1376 | "defaults": { 1377 | "color": { 1378 | "mode": "thresholds" 1379 | }, 1380 | "mappings": [], 1381 | "thresholds": { 1382 | "mode": "absolute", 1383 | "steps": [ 1384 | { 1385 | "color": "green" 1386 | }, 1387 | { 1388 | "color": "red", 1389 | "value": 80 1390 | } 1391 | ] 1392 | }, 1393 | "unit": "short" 1394 | }, 1395 | "overrides": [] 1396 | }, 1397 | "gridPos": { 1398 | "h": 3, 1399 | "w": 6, 1400 | "x": 12, 1401 | "y": 48 1402 | }, 1403 | "id": 16, 1404 | "options": { 1405 | "colorMode": "value", 1406 | "graphMode": "area", 1407 | "justifyMode": "auto", 1408 | "orientation": "auto", 1409 | "reduceOptions": { 1410 | "calcs": [ 1411 | "lastNotNull" 1412 | ], 1413 | "fields": "", 1414 | "values": false 1415 | }, 1416 | "textMode": "auto" 1417 | }, 1418 | "pluginVersion": "10.1.1", 1419 | "targets": [ 1420 | { 1421 | "datasource": { 1422 | "type": "prometheus", 1423 | "uid": "${DS_PROMETHEUS}" 1424 | }, 1425 | "disableTextWrap": false, 1426 | "editorMode": "code", 1427 | "expr": "increase(langchain_llm_tokens_total{agent_name=\"$agent_name\", ml_model_name=\"$model_name\", usage_type=\"total_tokens\"}[1h])", 1428 | "fullMetaSearch": false, 1429 | "includeNullMetadata": true, 1430 | "instant": false, 1431 | "legendFormat": "{{usage_type}}", 1432 | "range": true, 1433 | "refId": "A", 1434 | "useBackend": false 1435 | } 1436 | ], 1437 | "title": "Tokens used in last 1hr - total_tokens", 1438 | "type": "stat" 1439 | }, 1440 | { 1441 | "datasource": { 1442 | "type": "prometheus", 1443 | "uid": "${DS_PROMETHEUS}" 1444 | }, 1445 | "fieldConfig": { 1446 | "defaults": { 1447 | "color": { 1448 | "mode": "thresholds" 1449 | }, 1450 | "mappings": [], 1451 | "thresholds": { 1452 | "mode": "absolute", 1453 | "steps": [ 1454 | { 1455 | "color": "green" 1456 | }, 1457 | { 1458 | "color": "red", 1459 | "value": 80 1460 | } 1461 | ] 1462 | }, 1463 | "unit": "short" 1464 | }, 1465 | "overrides": [] 1466 | }, 1467 | "gridPos": { 1468 | "h": 3, 1469 | "w": 6, 1470 | "x": 18, 1471 | "y": 48 1472 | }, 1473 | "id": 18, 1474 | "options": { 1475 | "colorMode": "value", 1476 | "graphMode": "area", 1477 | "justifyMode": "auto", 1478 | "orientation": "auto", 1479 | "reduceOptions": { 1480 | "calcs": [ 1481 | "lastNotNull" 1482 | ], 1483 | "fields": "", 1484 | "values": false 1485 | }, 1486 | "textMode": "auto" 1487 | }, 1488 | "pluginVersion": "10.1.1", 1489 | "targets": [ 1490 | { 1491 | "datasource": { 1492 | "type": "prometheus", 1493 | "uid": "${DS_PROMETHEUS}" 1494 | }, 1495 | "disableTextWrap": false, 1496 | "editorMode": "code", 1497 | "expr": "increase(langchain_llm_tokens_total{agent_name=\"$agent_name\", ml_model_name=\"$model_name\", usage_type=\"completion_tokens\"}[1h])", 1498 | "fullMetaSearch": false, 1499 | "includeNullMetadata": true, 1500 | "instant": false, 1501 | "legendFormat": "{{usage_type}}", 1502 | "range": true, 1503 | "refId": "A", 1504 | "useBackend": false 1505 | } 1506 | ], 1507 | "title": "Tokens used in last 1hr - completion_tokens", 1508 | "type": "stat" 1509 | }, 1510 | { 1511 | "datasource": { 1512 | "type": "prometheus", 1513 | "uid": "${DS_PROMETHEUS}" 1514 | }, 1515 | "fieldConfig": { 1516 | "defaults": { 1517 | "color": { 1518 | "mode": "thresholds" 1519 | }, 1520 | "mappings": [], 1521 | "thresholds": { 1522 | "mode": "absolute", 1523 | "steps": [ 1524 | { 1525 | "color": "green" 1526 | }, 1527 | { 1528 | "color": "red", 1529 | "value": 80 1530 | } 1531 | ] 1532 | }, 1533 | "unit": "short" 1534 | }, 1535 | "overrides": [] 1536 | }, 1537 | "gridPos": { 1538 | "h": 3, 1539 | "w": 6, 1540 | "x": 12, 1541 | "y": 51 1542 | }, 1543 | "id": 17, 1544 | "options": { 1545 | "colorMode": "value", 1546 | "graphMode": "area", 1547 | "justifyMode": "auto", 1548 | "orientation": "auto", 1549 | "reduceOptions": { 1550 | "calcs": [ 1551 | "lastNotNull" 1552 | ], 1553 | "fields": "", 1554 | "values": false 1555 | }, 1556 | "textMode": "auto" 1557 | }, 1558 | "pluginVersion": "10.1.1", 1559 | "targets": [ 1560 | { 1561 | "datasource": { 1562 | "type": "prometheus", 1563 | "uid": "${DS_PROMETHEUS}" 1564 | }, 1565 | "disableTextWrap": false, 1566 | "editorMode": "code", 1567 | "expr": "increase(langchain_llm_tokens_total{agent_name=\"$agent_name\", ml_model_name=\"$model_name\", usage_type=\"prompt_tokens\"}[1h])", 1568 | "fullMetaSearch": false, 1569 | "includeNullMetadata": true, 1570 | "instant": false, 1571 | "legendFormat": "{{usage_type}}", 1572 | "range": true, 1573 | "refId": "A", 1574 | "useBackend": false 1575 | } 1576 | ], 1577 | "title": "Tokens used in last 1hr - prompt_tokens", 1578 | "type": "stat" 1579 | }, 1580 | { 1581 | "collapsed": false, 1582 | "gridPos": { 1583 | "h": 1, 1584 | "w": 24, 1585 | "x": 0, 1586 | "y": 56 1587 | }, 1588 | "id": 20, 1589 | "panels": [], 1590 | "title": "Langchain - Tools", 1591 | "type": "row" 1592 | }, 1593 | { 1594 | "datasource": { 1595 | "type": "prometheus", 1596 | "uid": "${DS_PROMETHEUS}" 1597 | }, 1598 | "fieldConfig": { 1599 | "defaults": { 1600 | "color": { 1601 | "mode": "palette-classic" 1602 | }, 1603 | "custom": { 1604 | "axisCenteredZero": false, 1605 | "axisColorMode": "text", 1606 | "axisLabel": "", 1607 | "axisPlacement": "auto", 1608 | "barAlignment": 0, 1609 | "drawStyle": "line", 1610 | "fillOpacity": 0, 1611 | "gradientMode": "none", 1612 | "hideFrom": { 1613 | "legend": false, 1614 | "tooltip": false, 1615 | "viz": false 1616 | }, 1617 | "insertNulls": false, 1618 | "lineInterpolation": "linear", 1619 | "lineWidth": 1, 1620 | "pointSize": 5, 1621 | "scaleDistribution": { 1622 | "type": "linear" 1623 | }, 1624 | "showPoints": "auto", 1625 | "spanNulls": false, 1626 | "stacking": { 1627 | "group": "A", 1628 | "mode": "none" 1629 | }, 1630 | "thresholdsStyle": { 1631 | "mode": "off" 1632 | } 1633 | }, 1634 | "mappings": [], 1635 | "thresholds": { 1636 | "mode": "absolute", 1637 | "steps": [ 1638 | { 1639 | "color": "green" 1640 | }, 1641 | { 1642 | "color": "red", 1643 | "value": 80 1644 | } 1645 | ] 1646 | }, 1647 | "unit": "short" 1648 | }, 1649 | "overrides": [] 1650 | }, 1651 | "gridPos": { 1652 | "h": 8, 1653 | "w": 12, 1654 | "x": 0, 1655 | "y": 57 1656 | }, 1657 | "id": 19, 1658 | "options": { 1659 | "legend": { 1660 | "calcs": [], 1661 | "displayMode": "list", 1662 | "placement": "bottom", 1663 | "showLegend": true 1664 | }, 1665 | "tooltip": { 1666 | "mode": "single", 1667 | "sort": "none" 1668 | } 1669 | }, 1670 | "targets": [ 1671 | { 1672 | "datasource": { 1673 | "type": "prometheus", 1674 | "uid": "${DS_PROMETHEUS}" 1675 | }, 1676 | "disableTextWrap": false, 1677 | "editorMode": "code", 1678 | "expr": "increase(langchain_tools_usage_total{agent_name=\"$agent_name\", execution_step=\"on_tool_start\"}[1m])", 1679 | "fullMetaSearch": false, 1680 | "includeNullMetadata": true, 1681 | "instant": false, 1682 | "legendFormat": "{{action}}", 1683 | "range": true, 1684 | "refId": "A", 1685 | "useBackend": false 1686 | } 1687 | ], 1688 | "title": "Langchain - Tool Usage", 1689 | "type": "timeseries" 1690 | }, 1691 | { 1692 | "datasource": { 1693 | "type": "prometheus", 1694 | "uid": "${DS_PROMETHEUS}" 1695 | }, 1696 | "fieldConfig": { 1697 | "defaults": { 1698 | "color": { 1699 | "mode": "palette-classic" 1700 | }, 1701 | "custom": { 1702 | "axisCenteredZero": false, 1703 | "axisColorMode": "text", 1704 | "axisLabel": "", 1705 | "axisPlacement": "auto", 1706 | "barAlignment": 0, 1707 | "drawStyle": "line", 1708 | "fillOpacity": 0, 1709 | "gradientMode": "none", 1710 | "hideFrom": { 1711 | "legend": false, 1712 | "tooltip": false, 1713 | "viz": false 1714 | }, 1715 | "insertNulls": false, 1716 | "lineInterpolation": "linear", 1717 | "lineWidth": 1, 1718 | "pointSize": 5, 1719 | "scaleDistribution": { 1720 | "type": "linear" 1721 | }, 1722 | "showPoints": "auto", 1723 | "spanNulls": false, 1724 | "stacking": { 1725 | "group": "A", 1726 | "mode": "none" 1727 | }, 1728 | "thresholdsStyle": { 1729 | "mode": "off" 1730 | } 1731 | }, 1732 | "mappings": [], 1733 | "thresholds": { 1734 | "mode": "absolute", 1735 | "steps": [ 1736 | { 1737 | "color": "green" 1738 | }, 1739 | { 1740 | "color": "red", 1741 | "value": 80 1742 | } 1743 | ] 1744 | }, 1745 | "unit": "s" 1746 | }, 1747 | "overrides": [] 1748 | }, 1749 | "gridPos": { 1750 | "h": 8, 1751 | "w": 12, 1752 | "x": 12, 1753 | "y": 57 1754 | }, 1755 | "id": 21, 1756 | "options": { 1757 | "legend": { 1758 | "calcs": [], 1759 | "displayMode": "list", 1760 | "placement": "bottom", 1761 | "showLegend": true 1762 | }, 1763 | "tooltip": { 1764 | "mode": "single", 1765 | "sort": "none" 1766 | } 1767 | }, 1768 | "targets": [ 1769 | { 1770 | "datasource": { 1771 | "type": "prometheus", 1772 | "uid": "${DS_PROMETHEUS}" 1773 | }, 1774 | "disableTextWrap": false, 1775 | "editorMode": "code", 1776 | "expr": "sum(rate(langchain_tool_latency_sum{agent_name=\"$agent_name\"}[1m])) / sum(rate(langchain_tool_latency_count{ agent_name=\"$agent_name\"}[1m]))\n", 1777 | "fullMetaSearch": false, 1778 | "includeNullMetadata": true, 1779 | "instant": false, 1780 | "legendFormat": "Average Latency", 1781 | "range": true, 1782 | "refId": "A", 1783 | "useBackend": false 1784 | }, 1785 | { 1786 | "datasource": { 1787 | "type": "prometheus", 1788 | "uid": "${DS_PROMETHEUS}" 1789 | }, 1790 | "editorMode": "code", 1791 | "expr": "histogram_quantile(0.90, sum(rate(langchain_tool_latency_bucket{agent_name=\"$agent_name\"}[1m])) by (le))\n", 1792 | "hide": false, 1793 | "instant": false, 1794 | "legendFormat": "p90", 1795 | "range": true, 1796 | "refId": "B" 1797 | }, 1798 | { 1799 | "datasource": { 1800 | "type": "prometheus", 1801 | "uid": "${DS_PROMETHEUS}" 1802 | }, 1803 | "editorMode": "code", 1804 | "expr": "histogram_quantile(0.95, sum(rate(langchain_tool_latency_bucket{agent_name=\"$agent_name\"}[1m])) by (le))\n", 1805 | "hide": false, 1806 | "instant": false, 1807 | "legendFormat": "p95", 1808 | "range": true, 1809 | "refId": "C" 1810 | }, 1811 | { 1812 | "datasource": { 1813 | "type": "prometheus", 1814 | "uid": "${DS_PROMETHEUS}" 1815 | }, 1816 | "editorMode": "code", 1817 | "expr": "histogram_quantile(0.99, sum(rate(langchain_tool_latency_bucket{agent_name=\"$agent_name\"}[1m])) by (le))\n", 1818 | "hide": false, 1819 | "instant": false, 1820 | "legendFormat": "p99", 1821 | "range": true, 1822 | "refId": "D" 1823 | } 1824 | ], 1825 | "title": "Langchain - Tool Latency", 1826 | "type": "timeseries" 1827 | }, 1828 | { 1829 | "datasource": { 1830 | "type": "prometheus", 1831 | "uid": "${DS_PROMETHEUS}" 1832 | }, 1833 | "fieldConfig": { 1834 | "defaults": { 1835 | "color": { 1836 | "mode": "palette-classic" 1837 | }, 1838 | "decimals": 0, 1839 | "mappings": [], 1840 | "thresholds": { 1841 | "mode": "absolute", 1842 | "steps": [ 1843 | { 1844 | "color": "green" 1845 | }, 1846 | { 1847 | "color": "red", 1848 | "value": 80 1849 | } 1850 | ] 1851 | }, 1852 | "unit": "percent" 1853 | }, 1854 | "overrides": [] 1855 | }, 1856 | "gridPos": { 1857 | "h": 8, 1858 | "w": 12, 1859 | "x": 0, 1860 | "y": 65 1861 | }, 1862 | "id": 13, 1863 | "options": { 1864 | "displayMode": "gradient", 1865 | "minVizHeight": 10, 1866 | "minVizWidth": 0, 1867 | "orientation": "auto", 1868 | "reduceOptions": { 1869 | "calcs": [ 1870 | "lastNotNull" 1871 | ], 1872 | "fields": "", 1873 | "values": false 1874 | }, 1875 | "showUnfilled": true, 1876 | "valueMode": "color" 1877 | }, 1878 | "pluginVersion": "10.1.1", 1879 | "targets": [ 1880 | { 1881 | "datasource": { 1882 | "type": "prometheus", 1883 | "uid": "${DS_PROMETHEUS}" 1884 | }, 1885 | "editorMode": "code", 1886 | "exemplar": false, 1887 | "expr": "sort(sum(increase(langchain_tool_latency_bucket{agent_name=\"$agent_name\", action=\"$tool_action\"}[1h])) by (le))", 1888 | "format": "time_series", 1889 | "hide": true, 1890 | "instant": true, 1891 | "legendFormat": "{{le}}", 1892 | "range": false, 1893 | "refId": "A" 1894 | }, 1895 | { 1896 | "datasource": { 1897 | "type": "prometheus", 1898 | "uid": "${DS_PROMETHEUS}" 1899 | }, 1900 | "editorMode": "code", 1901 | "exemplar": false, 1902 | "expr": "sort(\n 100 * \n increase(langchain_tool_latency_bucket{agent_name=\"$agent_name\",action=\"$tool_action\"}[1h]) \n / \n scalar(sum(increase(langchain_tool_latency_bucket{agent_name=\"$agent_name\",action=\"$tool_action\"}[1h])))\n)\n", 1903 | "hide": false, 1904 | "instant": true, 1905 | "legendFormat": "{{le}}s", 1906 | "range": false, 1907 | "refId": "B" 1908 | } 1909 | ], 1910 | "title": "Langchain - LLM Latency Distribution Past 1h", 1911 | "type": "bargauge" 1912 | }, 1913 | { 1914 | "datasource": { 1915 | "type": "prometheus", 1916 | "uid": "${DS_PROMETHEUS}" 1917 | }, 1918 | "fieldConfig": { 1919 | "defaults": { 1920 | "color": { 1921 | "mode": "palette-classic" 1922 | }, 1923 | "decimals": 0, 1924 | "mappings": [], 1925 | "thresholds": { 1926 | "mode": "absolute", 1927 | "steps": [ 1928 | { 1929 | "color": "green" 1930 | }, 1931 | { 1932 | "color": "red", 1933 | "value": 80 1934 | } 1935 | ] 1936 | }, 1937 | "unit": "percent" 1938 | }, 1939 | "overrides": [] 1940 | }, 1941 | "gridPos": { 1942 | "h": 8, 1943 | "w": 12, 1944 | "x": 12, 1945 | "y": 65 1946 | }, 1947 | "id": 23, 1948 | "options": { 1949 | "displayMode": "gradient", 1950 | "minVizHeight": 10, 1951 | "minVizWidth": 0, 1952 | "orientation": "auto", 1953 | "reduceOptions": { 1954 | "calcs": [ 1955 | "lastNotNull" 1956 | ], 1957 | "fields": "", 1958 | "values": false 1959 | }, 1960 | "showUnfilled": true, 1961 | "valueMode": "color" 1962 | }, 1963 | "pluginVersion": "10.1.1", 1964 | "targets": [ 1965 | { 1966 | "datasource": { 1967 | "type": "prometheus", 1968 | "uid": "${DS_PROMETHEUS}" 1969 | }, 1970 | "editorMode": "code", 1971 | "exemplar": false, 1972 | "expr": "sort(sum(increase(langchain_tool_latency_bucket{agent_name=\"$agent_name\", action=\"$tool_action\"}[3h])) by (le))", 1973 | "format": "time_series", 1974 | "hide": true, 1975 | "instant": true, 1976 | "legendFormat": "{{le}}", 1977 | "range": false, 1978 | "refId": "A" 1979 | }, 1980 | { 1981 | "datasource": { 1982 | "type": "prometheus", 1983 | "uid": "${DS_PROMETHEUS}" 1984 | }, 1985 | "editorMode": "code", 1986 | "exemplar": false, 1987 | "expr": "sort(\n 100 * \n increase(langchain_tool_latency_bucket{agent_name=\"$agent_name\",action=\"$tool_action\"}[3h]) \n / \n scalar(sum(increase(langchain_tool_latency_bucket{agent_name=\"$agent_name\",action=\"$tool_action\"}[3h])))\n)\n", 1988 | "hide": false, 1989 | "instant": true, 1990 | "legendFormat": "{{le}}s", 1991 | "range": false, 1992 | "refId": "B" 1993 | } 1994 | ], 1995 | "title": "Langchain - LLM Latency Distribution Past 1h", 1996 | "type": "bargauge" 1997 | }, 1998 | { 1999 | "datasource": { 2000 | "type": "prometheus", 2001 | "uid": "${DS_PROMETHEUS}" 2002 | }, 2003 | "fieldConfig": { 2004 | "defaults": { 2005 | "color": { 2006 | "mode": "thresholds" 2007 | }, 2008 | "mappings": [], 2009 | "thresholds": { 2010 | "mode": "absolute", 2011 | "steps": [ 2012 | { 2013 | "color": "green" 2014 | }, 2015 | { 2016 | "color": "red", 2017 | "value": 80 2018 | } 2019 | ] 2020 | }, 2021 | "unit": "short" 2022 | }, 2023 | "overrides": [] 2024 | }, 2025 | "gridPos": { 2026 | "h": 4, 2027 | "w": 6, 2028 | "x": 0, 2029 | "y": 73 2030 | }, 2031 | "id": 24, 2032 | "options": { 2033 | "colorMode": "value", 2034 | "graphMode": "area", 2035 | "justifyMode": "auto", 2036 | "orientation": "auto", 2037 | "reduceOptions": { 2038 | "calcs": [ 2039 | "lastNotNull" 2040 | ], 2041 | "fields": "", 2042 | "values": false 2043 | }, 2044 | "textMode": "auto" 2045 | }, 2046 | "pluginVersion": "10.1.1", 2047 | "targets": [ 2048 | { 2049 | "datasource": { 2050 | "type": "prometheus", 2051 | "uid": "${DS_PROMETHEUS}" 2052 | }, 2053 | "disableTextWrap": false, 2054 | "editorMode": "code", 2055 | "expr": "increase(langchain_tools_usage_total{execution_step=\"on_tool_end\", agent_name=\"$agent_name\", action=\"$tool_action\"}[1h])\n", 2056 | "fullMetaSearch": false, 2057 | "includeNullMetadata": true, 2058 | "instant": false, 2059 | "legendFormat": "{{usage_type}}", 2060 | "range": true, 2061 | "refId": "A", 2062 | "useBackend": false 2063 | } 2064 | ], 2065 | "title": "Tool Usage past 1hr - Success ", 2066 | "type": "stat" 2067 | }, 2068 | { 2069 | "datasource": { 2070 | "type": "prometheus", 2071 | "uid": "${DS_PROMETHEUS}" 2072 | }, 2073 | "fieldConfig": { 2074 | "defaults": { 2075 | "color": { 2076 | "mode": "thresholds" 2077 | }, 2078 | "mappings": [], 2079 | "thresholds": { 2080 | "mode": "absolute", 2081 | "steps": [ 2082 | { 2083 | "color": "green" 2084 | }, 2085 | { 2086 | "color": "red", 2087 | "value": 80 2088 | } 2089 | ] 2090 | }, 2091 | "unit": "short" 2092 | }, 2093 | "overrides": [] 2094 | }, 2095 | "gridPos": { 2096 | "h": 4, 2097 | "w": 6, 2098 | "x": 6, 2099 | "y": 73 2100 | }, 2101 | "id": 25, 2102 | "options": { 2103 | "colorMode": "value", 2104 | "graphMode": "area", 2105 | "justifyMode": "auto", 2106 | "orientation": "auto", 2107 | "reduceOptions": { 2108 | "calcs": [ 2109 | "lastNotNull" 2110 | ], 2111 | "fields": "", 2112 | "values": false 2113 | }, 2114 | "textMode": "auto" 2115 | }, 2116 | "pluginVersion": "10.1.1", 2117 | "targets": [ 2118 | { 2119 | "datasource": { 2120 | "type": "prometheus", 2121 | "uid": "${DS_PROMETHEUS}" 2122 | }, 2123 | "disableTextWrap": false, 2124 | "editorMode": "code", 2125 | "expr": "increase(langchain_tools_usage_total{execution_step=\"on_tool_error\", agent_name=\"$agent_name\", action=\"$tool_action\"}[1h])\n", 2126 | "fullMetaSearch": false, 2127 | "includeNullMetadata": true, 2128 | "instant": false, 2129 | "legendFormat": "{{usage_type}}", 2130 | "range": true, 2131 | "refId": "A", 2132 | "useBackend": false 2133 | } 2134 | ], 2135 | "title": "Tool Usage past 1hr - Errors ", 2136 | "type": "stat" 2137 | } 2138 | ], 2139 | "refresh": "", 2140 | "schemaVersion": 38, 2141 | "style": "dark", 2142 | "tags": [ 2143 | "langchain", 2144 | "vishwa-labs", 2145 | "vishwa.ai", 2146 | "mlmonitor" 2147 | ], 2148 | "templating": { 2149 | "list": [ 2150 | { 2151 | "current": {}, 2152 | "datasource": { 2153 | "type": "prometheus", 2154 | "uid": "${DS_PROMETHEUS}" 2155 | }, 2156 | "definition": "label_values(agent_name)", 2157 | "hide": 0, 2158 | "includeAll": false, 2159 | "label": "agent_name", 2160 | "multi": false, 2161 | "name": "agent_name", 2162 | "options": [], 2163 | "query": { 2164 | "query": "label_values(agent_name)", 2165 | "refId": "PrometheusVariableQueryEditor-VariableQuery" 2166 | }, 2167 | "refresh": 1, 2168 | "regex": "", 2169 | "skipUrlSync": false, 2170 | "sort": 0, 2171 | "type": "query" 2172 | }, 2173 | { 2174 | "current": {}, 2175 | "datasource": { 2176 | "type": "prometheus", 2177 | "uid": "${DS_PROMETHEUS}" 2178 | }, 2179 | "definition": "label_values(langchain_chain_execution_total{agent_name=\"$agent_name\"},ml_model_type)", 2180 | "hide": 0, 2181 | "includeAll": false, 2182 | "label": "model_type", 2183 | "multi": false, 2184 | "name": "model_type", 2185 | "options": [], 2186 | "query": { 2187 | "query": "label_values(langchain_chain_execution_total{agent_name=\"$agent_name\"},ml_model_type)", 2188 | "refId": "PrometheusVariableQueryEditor-VariableQuery" 2189 | }, 2190 | "refresh": 1, 2191 | "regex": "", 2192 | "skipUrlSync": false, 2193 | "sort": 0, 2194 | "type": "query" 2195 | }, 2196 | { 2197 | "current": {}, 2198 | "datasource": { 2199 | "type": "prometheus", 2200 | "uid": "${DS_PROMETHEUS}" 2201 | }, 2202 | "definition": "label_values(langchain_chain_execution_total{ml_model_type=\"$model_type\"},ml_model_name)", 2203 | "hide": 0, 2204 | "includeAll": false, 2205 | "label": "model_name", 2206 | "multi": false, 2207 | "name": "model_name", 2208 | "options": [], 2209 | "query": { 2210 | "query": "label_values(langchain_chain_execution_total{ml_model_type=\"$model_type\"},ml_model_name)", 2211 | "refId": "PrometheusVariableQueryEditor-VariableQuery" 2212 | }, 2213 | "refresh": 1, 2214 | "regex": "", 2215 | "skipUrlSync": false, 2216 | "sort": 0, 2217 | "type": "query" 2218 | }, 2219 | { 2220 | "current": {}, 2221 | "datasource": { 2222 | "type": "prometheus", 2223 | "uid": "${DS_PROMETHEUS}" 2224 | }, 2225 | "definition": "label_values(langchain_tools_usage_total{agent_name=\"$agent_name\"},action)", 2226 | "hide": 0, 2227 | "includeAll": false, 2228 | "label": "tool_action", 2229 | "multi": false, 2230 | "name": "tool_action", 2231 | "options": [], 2232 | "query": { 2233 | "query": "label_values(langchain_tools_usage_total{agent_name=\"$agent_name\"},action)", 2234 | "refId": "PrometheusVariableQueryEditor-VariableQuery" 2235 | }, 2236 | "refresh": 1, 2237 | "regex": "", 2238 | "skipUrlSync": false, 2239 | "sort": 0, 2240 | "type": "query" 2241 | } 2242 | ] 2243 | }, 2244 | "time": { 2245 | "from": "now-6h", 2246 | "to": "now" 2247 | }, 2248 | "timepicker": {}, 2249 | "timezone": "", 2250 | "title": "Langchain Observability Dashboard", 2251 | "uid": "e6473e10-af22-4fc3-bd1c-fbeb1637250f", 2252 | "version": 40, 2253 | "weekStart": "" 2254 | } 2255 | -------------------------------------------------------------------------------- /demo.py: -------------------------------------------------------------------------------- 1 | # from demo.openai_langchain import run_openai_agent 2 | # 3 | # res = run_openai_agent() 4 | # print(str(res)) 5 | 6 | from demo.mockgpt_runnable_langchain import run_openai_agent 7 | 8 | res = run_openai_agent() 9 | print(str(res)) 10 | -------------------------------------------------------------------------------- /demo/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vishwa-labs/vishwa-ml-sdk/0491d3c5a1e2301643e1fc8e0f18cfaf503b91c7/demo/__init__.py -------------------------------------------------------------------------------- /demo/fakechat_langchain.py: -------------------------------------------------------------------------------- 1 | import logging 2 | import os 3 | 4 | import openai 5 | from langchain.agents import initialize_agent, AgentType 6 | from langchain.chat_models import ChatOpenAI, FakeListChatModel 7 | from langchain.memory import ConversationBufferMemory 8 | 9 | from vishwa.mlmonitor.langchain.decorators.map_xpuls_project import MapXpulsProject 10 | from vishwa.mlmonitor.langchain.decorators.telemetry_override_labels import TelemetryOverrideLabels 11 | from vishwa.mlmonitor.langchain.instrument import LangchainTelemetry 12 | 13 | logger = logging.getLogger(__name__) 14 | 15 | 16 | # Set this to enable Advanced prompt tracing with server 17 | # os.environ["VISHWA_TRACING_ENABLED"] = "false" 18 | os.environ["VISHWA_TRACING_ENABLED"] = "false" 19 | 20 | default_labels = {"system": "openai-ln-test", "agent_name": "fallback_value"} 21 | 22 | LangchainTelemetry( 23 | default_labels=default_labels, 24 | xpuls_host_url="http://localhost:8000" 25 | ).auto_instrument() 26 | 27 | memory = ConversationBufferMemory(memory_key="chat_history") 28 | chat_model = FakeListChatModel( 29 | responses=[ 30 | "The Symphony of Ecosystems: Nature works through a delicate symphony of ecosystems, where every organism plays a critical role. The balance of life is like a finely tuned orchestra, with each species contributing to the overall harmony of the environment.", 31 | "Nature's Web of Interdependence: In nature, everything is interconnected. The balance of life is a web of interdependence, where the survival of one species often hinges on the well-being of another.", 32 | "The Dynamic Dance of Evolution: Nature operates through the dance of evolution, constantly adapting and evolving. The balance of life is not static but a dynamic equilibrium, ever-changing and adapting to new challenges.", 33 | "The Cycle of Renewal: Nature functions through cycles of growth, decay, and renewal. This cycle ensures the balance of life, as new life emerges from the old, maintaining a continuous flow of energy and resources. ", 34 | "The Balance of Opposites: Nature maintains balance through the interplay of opposites – day and night, predator and prey, growth and decay. This balance is the essence of life, creating a harmonious and sustainable world.", 35 | "The Gaia Hypothesis: Like the Gaia Hypothesis, nature can be seen as a self-regulating entity, where the balance of life is maintained through complex interactions among living organisms and their environment.", 36 | "Chaos and Order in Natural Systems: Nature operates on a spectrum from chaos to order. The balance of life lies in this spectrum, where seemingly chaotic natural events lead to the emergence of complex, ordered systems.", 37 | "The Ripple Effect of Actions: In nature, every action has a ripple effect. The balance of life is a testament to how small changes in one part of the ecosystem can have far-reaching impacts. ", 38 | "Nature as a Teacher: Nature works as the greatest teacher, showing us the importance of balance. The balance of life is a lesson in coexistence, sustainability, and respect for all living things", 39 | "The Mosaic of Biodiversity: Nature thrives on biodiversity. The balance of life is like a mosaic, where the variety of species creates a resilient and vibrant ecosystem." 40 | ] 41 | ) 42 | 43 | 44 | @TelemetryOverrideLabels(agent_name="chat_agent_alpha") 45 | @MapXpulsProject(project_id="default") # Get Project ID from console 46 | def run_fakechat_agent(): 47 | agent = initialize_agent(llm=chat_model, 48 | verbose=True, 49 | tools=[], 50 | agent=AgentType.CONVERSATIONAL_REACT_DESCRIPTION, 51 | memory=memory, 52 | # handle_parsing_errors="Check your output and make sure it conforms!", 53 | return_intermediate_steps=False, 54 | agent_executor_kwargs={"extra_prompt_messages": "test"}) 55 | 56 | try: 57 | res = agent.run("You are to behave as a think tank to answer the asked question in most creative way," 58 | " ensure to NOT be abusive or racist, you should validate your response w.r.t to validity " 59 | "in practical world before giving final answer" + 60 | f"\nQuestion: How does nature work?, is balance of life true? \n") 61 | except ValueError as e: 62 | res = str(e) 63 | if not res.startswith("Could not parse LLM output: `"): 64 | raise e 65 | logger.error(f" Got ValueError: {e}") 66 | res = res.removeprefix("Could not parse LLM output: `").removesuffix("`") 67 | 68 | return res 69 | -------------------------------------------------------------------------------- /demo/fastapi_langchain/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vishwa-labs/vishwa-ml-sdk/0491d3c5a1e2301643e1fc8e0f18cfaf503b91c7/demo/fastapi_langchain/__init__.py -------------------------------------------------------------------------------- /demo/fastapi_langchain/local.sh: -------------------------------------------------------------------------------- 1 | uvicorn main:app --port 6000 --reload 2 | -------------------------------------------------------------------------------- /demo/fastapi_langchain/main.py: -------------------------------------------------------------------------------- 1 | import os 2 | 3 | from fastapi import FastAPI 4 | from starlette.middleware.gzip import GZipMiddleware 5 | from starlette_exporter import PrometheusMiddleware, handle_metrics 6 | 7 | from vishwa.mlmonitor.langchain.instrument import LangchainTelemetry 8 | 9 | service_name = "Vishwa AI demo service" 10 | app = FastAPI( 11 | title=service_name, 12 | description="", 13 | version="0.0.1" 14 | ) 15 | app.add_middleware(GZipMiddleware, minimum_size=1000) 16 | app.add_middleware(PrometheusMiddleware, app_name=service_name, group_paths=True, filter_unhandled_paths=True) 17 | app.add_route("/metrics", handle_metrics) # Metrics are published at this endpoint 18 | 19 | # Initialise vishwa.ai 20 | default_labels = {"service": "vishwai-demo-service", "namespace": "mlops", 21 | "agent_name": "not_found"} 22 | 23 | os.environ["VISHWA_TRACING_ENABLED"] = "true" # ENABLE THIS ONLY IF ADVANCED vishwa.ai MONITORING IS REQUIRED 24 | 25 | # xpuls_host_url is an optional parameter and required if `VISHWA_TRACING_ENABLED` is enabled 26 | LangchainTelemetry(default_labels=default_labels, xpuls_host_url='https://api.vishwa.ai').auto_instrument() 27 | 28 | @app.get('/healthcheck') 29 | def health_check(): 30 | return "ping!" 31 | -------------------------------------------------------------------------------- /demo/fastapi_langchain/requirements.txt: -------------------------------------------------------------------------------- 1 | fastapi 2 | starlette-exporter 3 | starlette 4 | uvicorn 5 | -------------------------------------------------------------------------------- /demo/mockgpt_langchain.py: -------------------------------------------------------------------------------- 1 | import logging 2 | import os 3 | 4 | import openai 5 | from langchain.agents import initialize_agent, AgentType 6 | from langchain.chat_models import ChatOpenAI 7 | from langchain.memory import ConversationBufferMemory 8 | 9 | from vishwa.mlmonitor.langchain.decorators.map_xpuls_project import MapXpulsProject 10 | from vishwa.mlmonitor.langchain.decorators.telemetry_override_labels import TelemetryOverrideLabels 11 | from vishwa.mlmonitor.langchain.instrument import LangchainTelemetry 12 | 13 | logger = logging.getLogger(__name__) 14 | 15 | openai.api_key="sk-td6piaoyjv4nw1j6s6vd8t2xldq6xns" 16 | openai.api_base="https://mockgpt.wiremockapi.cloud/v1" 17 | 18 | os.environ["OPENAI_API_BASE"] = "https://mockgpt.wiremockapi.cloud/v1" 19 | os.environ["OPENAI_API_KEY"] = "sk-td6piaoyjv4nw1j6s6vd8t2xldq6xns" 20 | 21 | 22 | # Set this to enable Advanced prompt tracing with server 23 | # os.environ["VISHWA_TRACING_ENABLED"] = "false" 24 | os.environ["VISHWA_TRACING_ENABLED"] = "false" 25 | 26 | default_labels = {"system": "openai-ln-test", "agent_name": "fallback_value"} 27 | 28 | LangchainTelemetry( 29 | default_labels=default_labels, 30 | xpuls_host_url="http://localhost:8000" 31 | ).auto_instrument() 32 | 33 | memory = ConversationBufferMemory(memory_key="chat_history") 34 | chat_model = ChatOpenAI( 35 | deployment_name="gpt35turbo", 36 | model_name="gpt-35-turbo", 37 | temperature=0 38 | ) 39 | 40 | 41 | @TelemetryOverrideLabels(agent_name="chat_agent_alpha") 42 | @MapXpulsProject(project_id="default") # Get Project ID from console 43 | def run_openai_agent(): 44 | agent = initialize_agent(llm=chat_model, 45 | verbose=True, 46 | tools=[], 47 | agent=AgentType.CONVERSATIONAL_REACT_DESCRIPTION, 48 | memory=memory, 49 | # handle_parsing_errors="Check your output and make sure it conforms!", 50 | return_intermediate_steps=False, 51 | agent_executor_kwargs={"extra_prompt_messages": "test"}) 52 | 53 | try: 54 | res = agent.run("You are to behave as a think tank to answer the asked question in most creative way," 55 | " ensure to NOT be abusive or racist, you should validate your response w.r.t to validity " 56 | "in practical world before giving final answer" + 57 | f"\nQuestion: How does nature work?, is balance of life true? \n") 58 | except ValueError as e: 59 | res = str(e) 60 | if not res.startswith("Could not parse LLM output: `"): 61 | raise e 62 | logger.error(f" Got ValueError: {e}") 63 | res = res.removeprefix("Could not parse LLM output: `").removesuffix("`") 64 | 65 | return res 66 | -------------------------------------------------------------------------------- /demo/mockgpt_runnable_langchain.py: -------------------------------------------------------------------------------- 1 | import logging 2 | import os 3 | 4 | import openai 5 | from langchain.chat_models import AzureChatOpenAI 6 | 7 | import vishwa 8 | from vishwa.mlmonitor.langchain.decorators.map_xpuls_project import MapXpulsProject 9 | from vishwa.mlmonitor.langchain.decorators.telemetry_override_labels import TelemetryOverrideLabels 10 | from vishwa.mlmonitor.langchain.instrument import LangchainTelemetry 11 | from vishwa.mlmonitor.langchain.patches.xp_prompt_template import XPChatPromptTemplate 12 | from vishwa.prompt_hub import PromptClient 13 | 14 | logger = logging.getLogger(__name__) 15 | 16 | openai.api_key = os.getenv("OPENAI_API_KEY") 17 | openai.api_type = "azure" 18 | openai.api_base = os.getenv("OPENAI_URL") 19 | os.environ["OPENAI_API_BASE"] = os.getenv("OPENAI_URL") 20 | os.environ["OPENAI_API_VERSION"] = "2023-03-15-preview" 21 | openai.api_version = "2023-03-15-preview" 22 | 23 | # Set this to enable Advanced prompt tracing with server 24 | 25 | 26 | default_labels = {"system": "openai-ln-test", "agent_name": "fallback_value"} 27 | vishwa.host_url = "https://test-api.vishwa.ai" 28 | vishwa.api_key = "****************************************" 29 | vishwa.adv_tracing_enabled = "true" 30 | 31 | LangchainTelemetry( 32 | default_labels=default_labels, 33 | ).auto_instrument() 34 | 35 | chat_model = AzureChatOpenAI( 36 | deployment_name="gpt35turbo", 37 | model_name="gpt-35-turbo", 38 | temperature=0 39 | ) 40 | 41 | prompt_client = PromptClient( 42 | prompt_id="clrfm4v70jnlb1kph240", 43 | environment_name="dev" 44 | ) 45 | @TelemetryOverrideLabels(agent_name="chat_agent_alpha") 46 | @MapXpulsProject(project_id="defaultoPIt9USSR") # Get Project ID from console 47 | def run_openai_agent(): 48 | # prompt = ChatPromptTemplate.from_template("tell me a joke about {foo}") 49 | data = prompt_client.get_prompt({"variable-1": "I'm the first variable"}) 50 | prompt = XPChatPromptTemplate.from_template(data) 51 | chain = prompt | chat_model 52 | try: 53 | res = chain.invoke({"foo": "bears"}) 54 | except ValueError as e: 55 | res = str(e) 56 | if not res.startswith("Could not parse LLM output: `"): 57 | raise e 58 | logger.error(f" Got ValueError: {e}") 59 | res = res.removeprefix("Could not parse LLM output: `").removesuffix("`") 60 | 61 | return res 62 | -------------------------------------------------------------------------------- /demo/openai_langchain.py: -------------------------------------------------------------------------------- 1 | import logging 2 | import os 3 | 4 | import openai 5 | from langchain.agents import initialize_agent, AgentType 6 | from langchain.chat_models import AzureChatOpenAI 7 | from langchain.memory import ConversationBufferMemory 8 | 9 | from vishwa.mlmonitor.langchain.decorators.map_xpuls_project import MapXpulsProject 10 | from vishwa.mlmonitor.langchain.decorators.telemetry_override_labels import TelemetryOverrideLabels 11 | from vishwa.mlmonitor.langchain.instrument import LangchainTelemetry 12 | import vishwa 13 | from vishwa.prompt_hub import PromptClient 14 | 15 | logger = logging.getLogger(__name__) 16 | 17 | openai.api_key = os.getenv("OPENAI_API_KEY") 18 | openai.api_type = "azure" 19 | openai.api_base = os.getenv("OPENAI_URL") 20 | os.environ["OPENAI_API_BASE"] = os.getenv("OPENAI_URL") 21 | os.environ["OPENAI_API_VERSION"] = "2023-03-15-preview" 22 | openai.api_version = "2023-03-15-preview" 23 | 24 | # Set this to enable Advanced prompt tracing with server 25 | default_labels = {"system": "openai-ln-test", "agent_name": "fallback_value"} 26 | 27 | vishwa.host_url = "https://test-api.vishwa.ai" 28 | vishwa.api_key = "****************************************" 29 | vishwa.adv_tracing_enabled = "true" 30 | LangchainTelemetry(default_labels=default_labels).auto_instrument() 31 | 32 | memory = ConversationBufferMemory(memory_key="chat_history") 33 | chat_model = AzureChatOpenAI( 34 | deployment_name="gpt35turbo", 35 | model_name="gpt-35-turbo", 36 | temperature=0 37 | ) 38 | prompt = PromptClient( 39 | prompt_id="clrfm4v70jnlb1kph240", 40 | environment_name="dev" 41 | ) 42 | 43 | 44 | @TelemetryOverrideLabels(agent_name="chat_agent_alpha") 45 | @MapXpulsProject(project_slug="defaultoPIt9USSR") # Get Project ID from console 46 | def run_openai_agent(): 47 | agent = initialize_agent(llm=chat_model, 48 | verbose=True, 49 | tools=[], 50 | agent=AgentType.CONVERSATIONAL_REACT_DESCRIPTION, 51 | memory=memory, 52 | # handle_parsing_errors="Check your output and make sure it conforms!", 53 | return_intermediate_steps=False, 54 | agent_executor_kwargs={"extra_prompt_messages": "test"}) 55 | 56 | try: 57 | data = prompt.get_prompt({"variable-1": "I'm the first variable"}) 58 | res = agent.run(data.prompt) 59 | except ValueError as e: 60 | res = str(e) 61 | if not res.startswith("Could not parse LLM output: `"): 62 | raise e 63 | logger.error(f" Got ValueError: {e}") 64 | res = res.removeprefix("Could not parse LLM output: `").removesuffix("`") 65 | 66 | return res 67 | -------------------------------------------------------------------------------- /demo_requirements.txt: -------------------------------------------------------------------------------- 1 | openai 2 | langchain -------------------------------------------------------------------------------- /docs/images/langchain/langchain-dashboard-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vishwa-labs/vishwa-ml-sdk/0491d3c5a1e2301643e1fc8e0f18cfaf503b91c7/docs/images/langchain/langchain-dashboard-1.png -------------------------------------------------------------------------------- /docs/images/langchain/langchain-dashboard-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vishwa-labs/vishwa-ml-sdk/0491d3c5a1e2301643e1fc8e0f18cfaf503b91c7/docs/images/langchain/langchain-dashboard-2.png -------------------------------------------------------------------------------- /docs/images/langchain/langchain-dashboard-3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vishwa-labs/vishwa-ml-sdk/0491d3c5a1e2301643e1fc8e0f18cfaf503b91c7/docs/images/langchain/langchain-dashboard-3.png -------------------------------------------------------------------------------- /docs/images/langchain/langchain-dashboard-4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vishwa-labs/vishwa-ml-sdk/0491d3c5a1e2301643e1fc8e0f18cfaf503b91c7/docs/images/langchain/langchain-dashboard-4.png -------------------------------------------------------------------------------- /docs/langchain.md: -------------------------------------------------------------------------------- 1 | 2 | ## Langchain Documentation 3 | 4 | ### Basic Usage 5 | 6 | ```python 7 | from vishwa.mlmonitor.langchain.instrument import LangchainTelemetry 8 | 9 | # Add default labels that will be added to all captured metrics 10 | default_labels = {"service": "ml-project-service", "k8s-cluster": "app0", "namespace": "dev", 11 | "agent_name": "fallback_value"} 12 | 13 | # Enable the auto-telemetry 14 | LangchainTelemetry(default_labels=default_labels).auto_instrument() 15 | 16 | ``` 17 | 18 | ### Advanced Guide 19 | #### 1. Decorator for overriding default labels 20 | Can be used in LLM Apps which have multi-agent in the workflow [Optional] 21 | 22 | Only labels defined can be overriden, if you wish you add a new label, then it needs to defined in `default_labels` 23 | ```python 24 | # Overriding value `agent_nam`e defined in `default_labels` 25 | @TelemetryOverrideLabels(agent_name="chat_agent_alpha") # `agent_name` here is overriden for the scope of this function 26 | def get_response_using_agent_alpha(prompt, query): 27 | agent = initialize_agent(llm=chat_model, 28 | verbose=True, 29 | agent=CONVERSATIONAL_REACT_DESCRIPTION, 30 | memory=memory) 31 | 32 | res = agent.run(f"{prompt}. \n Query: {query}") 33 | ``` 34 | 35 | ## Monitoring 36 | 37 | We have created a template grafana dashboard setup for you to get started. 38 | 39 | You can find the dashboard template here -> [grafana template](../dashboards/grafana_langchain.json) 40 | 41 | ### Screenshots 42 | 43 | | ![langchain-dashboard-1](images/langchain/langchain-dashboard-1.png) | ![langchain-dashboard-2](images/langchain/langchain-dashboard-2.png) | 44 | |---|---| 45 | | ![langchain-dashboard-3](images/langchain/langchain-dashboard-3.png) | ![langchain-dashboard-4](images/langchain/langchain-dashboard-4.png) | 46 | 47 | 48 | `Note`: "No Data" for few fields in the screenshot is because of unavailability of data at the point of taking the screenshot, so it shouldn't be an issue. -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | -r requirements/base_requirements.txt 2 | -r requirements/requirements_langchain.txt 3 | -------------------------------------------------------------------------------- /requirements/base_requirements.txt: -------------------------------------------------------------------------------- 1 | prometheus-client 2 | pydantic 3 | requests 4 | urllib3 5 | -------------------------------------------------------------------------------- /requirements/requirements_langchain.txt: -------------------------------------------------------------------------------- 1 | -r base_requirements.txt 2 | langchain 3 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup, find_packages 2 | import os 3 | 4 | 5 | def read_requirements(file_name): 6 | file_name = os.path.abspath(file_name) # Convert to absolute path 7 | requirements = [] 8 | with open(file_name, 'r') as file: 9 | for line in file: 10 | line = line.strip() 11 | if line.startswith('-r') or line.startswith('--requirement'): 12 | # Recursively read referenced requirements file 13 | referenced_file = line.split(maxsplit=1)[1] 14 | requirements.extend(read_requirements(os.path.join(os.path.dirname(file_name), referenced_file))) 15 | elif line and not line.startswith('#'): # Ignore comment lines 16 | requirements.append(line) 17 | return requirements 18 | 19 | 20 | with open("README.md", "r", encoding="utf-8") as fh: 21 | long_description = fh.read() 22 | 23 | setup( 24 | name='vishwa-ml-sdk', 25 | version='0.4.0', 26 | author='Sai Sharan Tangeda', 27 | author_email='saisarantangeda@gmail.com', 28 | description='Integration SDK for vishwa.ai', 29 | license='Apache License 2.0', 30 | url='https://github.com/vishwa-labs/vishwa-ml-sdk', 31 | packages=find_packages(), 32 | install_requires=read_requirements('requirements.txt'), 33 | extras_require={ 34 | 'langchain': read_requirements('requirements/requirements_langchain.txt'), 35 | 'all': read_requirements('requirements.txt') 36 | }, 37 | long_description_content_type='text/markdown', 38 | long_description=long_description, 39 | ) 40 | -------------------------------------------------------------------------------- /vishwa/__init__.py: -------------------------------------------------------------------------------- 1 | import os 2 | 3 | 4 | api_key = os.environ.get("VISHWA_API_KEY") 5 | host_url = os.environ.get("VISHWA_HOST_URL", "https://api.vishwa.ai") 6 | adv_tracing_enabled = os.environ.get("VISHWA_TRACING_ENABLED", "false") 7 | -------------------------------------------------------------------------------- /vishwa/client/__init__.py: -------------------------------------------------------------------------------- 1 | from .xpuls_client import XpulsAIClient -------------------------------------------------------------------------------- /vishwa/client/constants.py: -------------------------------------------------------------------------------- 1 | VISHWA_API_KEY = "VISHWA_API_KEY" 2 | VISHWA_HOST_URL = "VISHWA_HOST_URL" 3 | VISHWA_TRACING_ENABLED = "VISHWA_TRACING_ENABLED" 4 | -------------------------------------------------------------------------------- /vishwa/client/models.py: -------------------------------------------------------------------------------- 1 | from typing import List 2 | 3 | from pydantic import BaseModel 4 | 5 | 6 | class PromptVariable(BaseModel): 7 | variable: str 8 | default: str 9 | 10 | 11 | class PromptResponseData(BaseModel): 12 | prompt_version_id: str 13 | prompt_id: str 14 | prompt_external_id: str 15 | prompt: str 16 | prompt_variables: List[PromptVariable] 17 | 18 | 19 | class XPPrompt(BaseModel): 20 | prompt_version_id: str 21 | prompt_id: str 22 | prompt_external_id: str 23 | prompt: str 24 | 25 | -------------------------------------------------------------------------------- /vishwa/client/xpuls_client.py: -------------------------------------------------------------------------------- 1 | import os 2 | from typing import Optional 3 | 4 | import requests 5 | import time 6 | import logging 7 | 8 | import vishwa 9 | from vishwa.client import constants 10 | from vishwa.client.models import PromptResponseData 11 | 12 | 13 | class XpulsAIClient: 14 | def __init__(self): 15 | self._host_url = vishwa.host_url 16 | self._api_key = vishwa.api_key 17 | 18 | self._headers = {"XP-API-Key": self._api_key} 19 | 20 | def _make_request_with_retries(self, endpoint, method='GET', data=None, retries=3, backoff_factor=2): 21 | """ 22 | Make an API request with auto-retries and crashloop backoff. 23 | Supports GET, POST, PUT, and DELETE requests. 24 | """ 25 | if endpoint.startswith("/"): 26 | url = f"{self._host_url}/{endpoint[1:]}" 27 | else: 28 | url = f"{self._host_url}/{endpoint}" 29 | for attempt in range(retries): 30 | try: 31 | if method == 'GET': 32 | response = requests.get(url, headers=self._headers) 33 | elif method == 'POST': 34 | response = requests.post(url, headers=self._headers, json=data) 35 | elif method == 'PUT': 36 | response = requests.put(url, headers=self._headers, json=data) 37 | elif method == 'DELETE': 38 | response = requests.delete(url, headers=self._headers) 39 | else: 40 | raise ValueError("Unsupported HTTP method") 41 | 42 | response.raise_for_status() 43 | return response.json() 44 | except requests.RequestException as e: 45 | logging.warning(f"Request failed with error {e}, attempt {attempt + 1} of {retries}") 46 | time.sleep(backoff_factor ** attempt) 47 | 48 | raise Exception("Max retries exceeded") 49 | 50 | def get_live_prompt(self, prompt_id: str, env_name: str) -> PromptResponseData: 51 | data = self._make_request_with_retries( 52 | endpoint=f"/v1/prompt/{prompt_id}/env/{env_name}", 53 | method="GET", 54 | 55 | ) 56 | 57 | return PromptResponseData(**data) 58 | -------------------------------------------------------------------------------- /vishwa/mlmonitor/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vishwa-labs/vishwa-ml-sdk/0491d3c5a1e2301643e1fc8e0f18cfaf503b91c7/vishwa/mlmonitor/__init__.py -------------------------------------------------------------------------------- /vishwa/mlmonitor/langchain/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vishwa-labs/vishwa-ml-sdk/0491d3c5a1e2301643e1fc8e0f18cfaf503b91c7/vishwa/mlmonitor/langchain/__init__.py -------------------------------------------------------------------------------- /vishwa/mlmonitor/langchain/decorators/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vishwa-labs/vishwa-ml-sdk/0491d3c5a1e2301643e1fc8e0f18cfaf503b91c7/vishwa/mlmonitor/langchain/decorators/__init__.py -------------------------------------------------------------------------------- /vishwa/mlmonitor/langchain/decorators/map_xpuls_project.py: -------------------------------------------------------------------------------- 1 | import contextvars 2 | from typing import Optional, Any, Dict 3 | 4 | 5 | class MapXpulsProject: 6 | _context: contextvars.ContextVar[Optional[Dict[str, Any]]] = contextvars.ContextVar('telemetry_extra_labels_vars', 7 | default=None) 8 | 9 | def __init__(self, project_id: Optional[str] = None, project_slug: Optional[str] = None): 10 | if project_id is None and project_slug is None: 11 | raise ValueError("Both `project_id` and `project_slug` cannot be null") 12 | self.project_id = project_id 13 | self.project_slug = project_slug 14 | 15 | def __call__(self, func): 16 | def wrapped_func(*args, **kwargs): 17 | self._context.set({'project_id': self.project_id, 'project_slug': self.project_slug}) 18 | 19 | return func(*args, **kwargs) 20 | 21 | return wrapped_func 22 | -------------------------------------------------------------------------------- /vishwa/mlmonitor/langchain/decorators/telemetry_override_labels.py: -------------------------------------------------------------------------------- 1 | import contextvars 2 | from typing import Optional, Any, Dict 3 | 4 | 5 | class TelemetryOverrideLabels: 6 | _context: contextvars.ContextVar[Optional[Dict[str, Any]]] = contextvars.ContextVar('telemetry_extra_labels_vars', 7 | default=None) 8 | 9 | def __init__(self, **labels): 10 | self.labels = labels 11 | 12 | def __call__(self, func): 13 | def wrapped_func(*args, **kwargs): 14 | self._context.set(self.labels) 15 | 16 | return func(*args, **kwargs) 17 | 18 | return wrapped_func 19 | -------------------------------------------------------------------------------- /vishwa/mlmonitor/langchain/handlers/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vishwa-labs/vishwa-ml-sdk/0491d3c5a1e2301643e1fc8e0f18cfaf503b91c7/vishwa/mlmonitor/langchain/handlers/__init__.py -------------------------------------------------------------------------------- /vishwa/mlmonitor/langchain/handlers/callback_handlers.py: -------------------------------------------------------------------------------- 1 | import time 2 | import logging 3 | from typing import Any, Dict, List, Union 4 | 5 | import pydantic 6 | from langchain.callbacks.base import AsyncCallbackHandler 7 | from langchain.schema.output import LLMResult 8 | from langchain.schema.messages import BaseMessage 9 | from langchain.schema.agent import AgentAction, AgentFinish 10 | 11 | from vishwa.mlmonitor.langchain.profiling.prometheus import LangchainChainMetrics, LangchainPrometheusMetrics, \ 12 | LangchainChatModelMetrics, LLMTokens, LangchainToolMetrics 13 | from vishwa.mlmonitor.utils.common import get_safe_dict_value 14 | 15 | from . import constants as c 16 | # Set the tracer provider and a console exporter 17 | 18 | 19 | class CallbackHandler(AsyncCallbackHandler): 20 | log = logging.getLogger() 21 | 22 | def __init__(self, ln_metrics: LangchainPrometheusMetrics, chain_run_id: str, 23 | override_labels: Dict[str, str]) -> None: 24 | self.llm_start_time = None 25 | self.llm_end_time = None 26 | 27 | self.chat_start_time = None 28 | self.chat_end_time = None 29 | self.chat_model_start_metrics = None 30 | 31 | self.tool_start_time = None 32 | self.tool_end_time = None 33 | self.tool_metrics = None 34 | 35 | self.ln_metrics = ln_metrics 36 | self.chain_run_id = chain_run_id 37 | 38 | self.chain_end_time = None 39 | self.chain_start_time = None 40 | self.chain_start_metrics = None 41 | 42 | self.override_labels = override_labels 43 | 44 | def _get_model_name(self, data, model_type): 45 | if model_type == c.WORD_CHAT_MODELS: 46 | model_info = get_safe_dict_value(get_safe_dict_value( 47 | get_safe_dict_value(data, 'kwargs', {}), 48 | 'llm', 49 | {} 50 | ), 'kwargs', {}) 51 | return model_info['model_name'] if 'model_name' in model_info else '' 52 | elif model_type == c.WORD_LLM: 53 | return 'default' ## TODO: Improve 54 | else: 55 | return 'default' 56 | 57 | def on_llm_start( 58 | self, serialized: Dict[str, Any], prompts: List[str], **kwargs: Any 59 | ) -> Any: 60 | """Run when LLM starts running.""" 61 | 62 | self.log.debug(f"on_llm_start, {serialized}, {prompts}, {kwargs}") 63 | 64 | def on_chat_model_start( 65 | self, serialized: Dict[str, Any], messages: List[List[BaseMessage]], **kwargs: Any 66 | ) -> Any: 67 | """Run when Chat Model starts running.""" 68 | 69 | tags = get_safe_dict_value(kwargs, 'tags') 70 | parent_run_id = get_safe_dict_value(kwargs, 'parent_run_id') 71 | ml_model_name = get_safe_dict_value(get_safe_dict_value(serialized, 'kwargs', {}), 72 | 'model_name', '') 73 | 74 | self.chat_model_start_metrics = LangchainChatModelMetrics( 75 | lc=str(get_safe_dict_value(serialized, 'lc')), 76 | type=get_safe_dict_value(serialized, 'type'), 77 | execution_step='on_chat_model_start', 78 | agent_type=tags[0] if len(tags) > 0 else c.WORD_UNDEFINED, 79 | ml_model_type='chat_models', 80 | ml_model_name=ml_model_name, 81 | other_tags=','.join(list(tags)), 82 | ) 83 | self.chat_start_time = time.time() # Record start time 84 | if self.llm_start_time is None: 85 | self.llm_start_time = time.time() # Record start time 86 | self.ln_metrics.add_chat_model_counter(self.chat_model_start_metrics, self.override_labels) 87 | 88 | self.log.debug(f"on_chat_model_start, {serialized}, {messages}, {kwargs}") 89 | 90 | def on_llm_new_token(self, token: str, **kwargs: Any) -> Any: 91 | """Run on new LLM token. Only available when streaming is enabled.""" 92 | self.log.debug(f"on_llm_new_token, {token}") 93 | 94 | def on_llm_end(self, response: LLMResult, **kwargs: Any) -> Any: 95 | """Run when LLM ends running.""" 96 | tags = get_safe_dict_value(kwargs, 'tags') 97 | token_usage = get_safe_dict_value(response.llm_output, 98 | 'token_usage') if response.llm_output is not None else None 99 | execution_step = "on_llm_end" 100 | if token_usage is not None: 101 | llm_tokens = LLMTokens( 102 | execution_step=execution_step, 103 | ml_model_type='llm', 104 | ml_model_name=get_safe_dict_value(response.llm_output, 105 | 'model_name') if response.llm_output is not None else "fake_chat_model", 106 | other_tags=','.join(list(tags)), 107 | ) 108 | self.ln_metrics.add_llm_tokens_usage(llm_tokens, 'prompt_tokens', 109 | int(dict(token_usage)['prompt_tokens']), self.override_labels) 110 | self.ln_metrics.add_llm_tokens_usage(llm_tokens, 'total_tokens', 111 | int(dict(token_usage)['total_tokens']), self.override_labels) 112 | self.ln_metrics.add_llm_tokens_usage(llm_tokens, 'completion_tokens', 113 | int(dict(token_usage)['completion_tokens']), self.override_labels) 114 | if self.chat_model_start_metrics is not None: 115 | 116 | llm_end_metrics = LangchainChatModelMetrics( 117 | lc=self.chat_model_start_metrics.lc, 118 | type=self.chat_model_start_metrics.type, 119 | execution_step=execution_step, 120 | agent_type=tags[0] if len(tags) > 0 else c.WORD_UNDEFINED, 121 | ml_model_type=self.chat_model_start_metrics.ml_model_type, 122 | ml_model_name=self.chat_model_start_metrics.ml_model_name, 123 | other_tags=','.join(list(tags)), 124 | ) 125 | elapsed_time = time.time() - self.chat_start_time # Record start time 126 | self.ln_metrics.add_chat_model_counter(llm_end_metrics, self.override_labels) 127 | self.ln_metrics.observe_chat_model_latency(llm_end_metrics, elapsed_time, self.override_labels) 128 | 129 | elif self.llm_start_time is not None: 130 | llm_end_metrics = LangchainChatModelMetrics( 131 | lc='-1', 132 | type=c.WORD_UNDEFINED, 133 | execution_step='on_llm_end', 134 | agent_type=tags[0] if len(tags) > 0 else c.WORD_UNDEFINED, 135 | ml_model_type=self.chat_model_start_metrics.ml_model_type, 136 | ml_model_name=self.chat_model_start_metrics.ml_model_name, 137 | other_tags=','.join(list(tags)), 138 | ) 139 | elapsed_time = time.time() - self.llm_start_time # Record start time 140 | self.ln_metrics.add_chat_model_counter(llm_end_metrics, self.override_labels) 141 | self.ln_metrics.observe_chat_model_latency(llm_end_metrics, elapsed_time, self.override_labels) 142 | 143 | self.log.debug(f"on_llm_end, {response}, {kwargs}") 144 | 145 | def on_llm_error( 146 | self, error: Union[Exception, KeyboardInterrupt], **kwargs: Any 147 | ) -> Any: 148 | """Run when LLM errors.""" 149 | self.log.debug("on_llm_error") 150 | 151 | def on_chain_start(self, serialized: Dict[str, Any], inputs: Dict[str, Any], **kwargs: Any) -> Any: 152 | """Run when chain starts running.""" 153 | tags = get_safe_dict_value(kwargs, 'tags') 154 | parent_run_id = get_safe_dict_value(kwargs, 'parent_run_id') 155 | model_type = 'chat_models' if 'chat_models' in get_safe_dict_value(get_safe_dict_value( 156 | get_safe_dict_value(serialized, 'kwargs', {}), 157 | 'llm', 158 | {} 159 | ), 'id', []) else 'llm' 160 | 161 | self.chain_start_metrics = LangchainChainMetrics( 162 | lc=str(get_safe_dict_value(serialized, 'lc')), 163 | type=get_safe_dict_value(serialized, 'type'), 164 | agent_type=tags[0] if len(tags) > 0 else c.WORD_UNDEFINED, 165 | # run_id=str(get_safe_dict_value(kwargs, 'run_id')), 166 | # chain_run_id=self.chain_run_id, 167 | # parent_run_id=str(parent_run_id) if parent_run_id is not None else None, 168 | ml_model_type=model_type, 169 | other_tags=','.join(list(tags)), 170 | ml_model_name=self._get_model_name(serialized, model_type), 171 | execution_step='on_chain_start', 172 | 173 | ) 174 | 175 | self.chain_start_time = time.time() # Record start time 176 | self.ln_metrics.add_chain_counter(self.chain_start_metrics, self.override_labels) 177 | 178 | self.log.debug(f"on_chain_start, {serialized}, {inputs}, {kwargs}") 179 | 180 | def on_chain_end(self, outputs: Dict[str, Any], **kwargs: Any) -> Any: 181 | """Run when chain ends running.""" 182 | output_chars = get_safe_dict_value(outputs, 'text') 183 | tags = get_safe_dict_value(kwargs, 'tags') 184 | parent_run_id = get_safe_dict_value(kwargs, 'parent_run_id') 185 | 186 | if self.chain_start_metrics is not None: 187 | if pydantic.__version__.startswith("2"): 188 | chain_end_metrics = self.chain_start_metrics.model_copy( 189 | update={'execution_step': 'on_chain_end'}, deep=True 190 | ) 191 | else: 192 | chain_end_metrics = self.chain_start_metrics.copy( 193 | update={'execution_step': 'on_chain_end'}, deep=True 194 | ) 195 | else: 196 | chain_end_metrics = LangchainChainMetrics( 197 | lc='-1', 198 | type=c.WORD_UNDEFINED, 199 | agent_type=tags[0] if len(tags) > 0 else c.WORD_UNDEFINED, 200 | ml_model_type='llm', 201 | other_tags=','.join(list(tags)), 202 | ml_model_name='undefined', 203 | execution_step='on_chain_end', 204 | 205 | ) 206 | elapsed_time = time.time() - self.chain_start_time 207 | 208 | self.ln_metrics.add_chain_counter(chain_end_metrics, self.override_labels) 209 | self.ln_metrics.observe_chain_latency(chain_end_metrics, elapsed_time, self.override_labels) 210 | self.log.debug(f"on_chain_end, {outputs}, {kwargs}, {self.chat_start_time}") 211 | 212 | def on_chain_error(self, error: Union[Exception, KeyboardInterrupt], **kwargs: Any) -> Any: 213 | """Run when chain errors.""" 214 | tags = get_safe_dict_value(kwargs, 'tags') 215 | parent_run_id = get_safe_dict_value(kwargs, 'parent_run_id') 216 | 217 | if self.chain_start_metrics is not None: 218 | if pydantic.__version__.startswith("2"): 219 | chain_err_metrics = self.chain_start_metrics.model_copy( 220 | update={'execution_step': 'on_chain_error'}, deep=True 221 | ) 222 | else: 223 | chain_err_metrics = self.chain_start_metrics.copy( 224 | update={'execution_step': 'on_chain_error'}, deep=True 225 | ) 226 | else: 227 | chain_err_metrics = LangchainChainMetrics( 228 | lc='-1', 229 | type=c.WORD_UNDEFINED, 230 | agent_type=tags[0] if len(tags) > 0 else c.WORD_UNDEFINED, 231 | # run_id=str(get_safe_dict_value(kwargs, 'run_id')), 232 | # parent_run_id=str(parent_run_id) if parent_run_id is not None else None, 233 | # chain_run_id=self.chain_run_id, 234 | ml_model_type='llm', 235 | other_tags=','.join(list(tags)), 236 | ml_model_name=c.WORD_UNDEFINED, 237 | execution_step='on_chain_error', 238 | 239 | ) 240 | 241 | elapsed_time = time.time() - self.chain_start_time 242 | 243 | self.ln_metrics.add_chain_counter(chain_err_metrics, self.override_labels) 244 | self.ln_metrics.observe_chain_latency(chain_err_metrics, elapsed_time, self.override_labels) 245 | self.log.debug("on_chain_error") 246 | 247 | def on_tool_start( 248 | self, serialized: Dict[str, Any], input_str: str, **kwargs: Any 249 | ) -> Any: 250 | """Run when tool starts running.""" 251 | self.tool_start_time = time.time() 252 | self.tool_metrics = LangchainToolMetrics( 253 | execution_step='on_tool_start', 254 | action=get_safe_dict_value(serialized, 'name'), 255 | # run_id=str(get_safe_dict_value(kwargs, 'run_id')), 256 | # parent_run_id=str(get_safe_dict_value(kwargs, 'parent_run_id')), 257 | agent_type=self.chain_start_metrics.agent_type, 258 | other_tags=','.join(get_safe_dict_value(kwargs, 'tags')) 259 | ) 260 | 261 | self.ln_metrics.add_tools_usage_counter(self.tool_metrics, self.override_labels) 262 | self.log.debug(f"on_tool_start, {serialized}, {input_str}, {kwargs}") 263 | 264 | def on_tool_end(self, output: str, **kwargs: Any) -> Any: 265 | """Run when tool ends running.""" 266 | 267 | tool_metrics = LangchainToolMetrics( 268 | execution_step='on_tool_end', 269 | action=self.tool_metrics.action, 270 | # run_id=str(get_safe_dict_value(kwargs, 'run_id')), 271 | # parent_run_id=str(get_safe_dict_value(kwargs, 'parent_run_id')), 272 | agent_type=self.chain_start_metrics.agent_type, 273 | other_tags=','.join(get_safe_dict_value(kwargs, 'tags')) 274 | ) 275 | 276 | elapsed_time = time.time() - self.tool_start_time 277 | 278 | self.ln_metrics.add_tools_usage_counter(tool_metrics, self.override_labels) 279 | self.ln_metrics.observe_tool_latency(tool_metrics, elapsed_time, self.override_labels) 280 | 281 | self.log.debug(f"on_tool_end, {output}") 282 | 283 | def on_tool_error( 284 | self, error: Union[Exception, KeyboardInterrupt], **kwargs: Any 285 | ) -> Any: 286 | """Run when tool errors.""" 287 | 288 | tool_metrics = LangchainToolMetrics( 289 | execution_step='on_tool_error', 290 | action=self.tool_metrics.action, 291 | # run_id=str(get_safe_dict_value(kwargs, 'run_id')), 292 | # parent_run_id=str(get_safe_dict_value(kwargs, 'parent_run_id')), 293 | agent_type=self.chain_start_metrics.agent_type, 294 | other_tags=','.join(get_safe_dict_value(kwargs, 'tags')) 295 | ) 296 | 297 | elapsed_time = time.time() - self.tool_start_time 298 | 299 | self.ln_metrics.add_tools_usage_counter(tool_metrics, self.override_labels) 300 | self.ln_metrics.observe_tool_latency(tool_metrics, elapsed_time, self.override_labels) 301 | 302 | self.log.debug("on_tool_error") 303 | 304 | def on_text(self, text: str, **kwargs: Any) -> Any: 305 | """Run on arbitrary text.""" 306 | pass 307 | 308 | def on_agent_action(self, action: AgentAction, **kwargs: Any) -> Any: 309 | """Run on agent action.""" 310 | self.log.debug(f"on_agent_action, {action}") 311 | 312 | def on_agent_finish(self, finish: AgentFinish, **kwargs: Any) -> Any: 313 | """Run on agent end.""" 314 | self.log.debug(f"on_agent_finish, {finish}") 315 | -------------------------------------------------------------------------------- /vishwa/mlmonitor/langchain/handlers/constants.py: -------------------------------------------------------------------------------- 1 | 2 | ## COMMON WORDS 3 | WORD_UNDEFINED = "undefined" 4 | WORD_CHAT_MODELS = "chat_models" 5 | WORD_LLM = "llm" 6 | -------------------------------------------------------------------------------- /vishwa/mlmonitor/langchain/instrument.py: -------------------------------------------------------------------------------- 1 | from typing import Dict, Any, Optional 2 | 3 | from vishwa.mlmonitor.langchain.patches import patch_run 4 | from vishwa.mlmonitor.langchain.patches.patch_invoke import patch_invoke 5 | from vishwa.mlmonitor.langchain.profiling.prometheus import LangchainPrometheusMetrics 6 | from vishwa.mlmonitor.langchain.xpuls_client import XpulsAILangChainClient 7 | 8 | 9 | class LangchainTelemetry: 10 | def __init__(self, default_labels: Dict[str, Any], 11 | enable_prometheus: bool = True,): 12 | self.ln_metrics = LangchainPrometheusMetrics(default_labels) 13 | 14 | self.xpuls_client = XpulsAILangChainClient() 15 | 16 | self.default_labels = default_labels 17 | self.enable_prometheus = enable_prometheus 18 | 19 | def auto_instrument(self): 20 | patch_run(self.ln_metrics, self.xpuls_client) 21 | patch_invoke(self.ln_metrics, self.xpuls_client) 22 | print("** ProfileML -> Langchain auto-instrumentation completed successfully **") 23 | -------------------------------------------------------------------------------- /vishwa/mlmonitor/langchain/patches/__init__.py: -------------------------------------------------------------------------------- 1 | from .patch_run import * 2 | -------------------------------------------------------------------------------- /vishwa/mlmonitor/langchain/patches/patch_invoke.py: -------------------------------------------------------------------------------- 1 | import json 2 | import os 3 | import uuid 4 | from typing import Optional, Dict, Any 5 | 6 | from langchain.callbacks import LangChainTracer 7 | from langchain.chains.base import Chain 8 | from langchain.load.dump import dumpd 9 | from langchain.schema.runnable import RunnableConfig, RunnableSequence 10 | from langchain.schema.runnable.base import Input 11 | from langchain.schema.runnable.config import ensure_config 12 | 13 | import vishwa 14 | from vishwa.mlmonitor.langchain.handlers.callback_handlers import CallbackHandler 15 | from vishwa.mlmonitor.langchain.patches.utils import get_scoped_override_labels, get_scoped_project_info 16 | from vishwa.mlmonitor.langchain.profiling.prometheus import LangchainPrometheusMetrics 17 | from vishwa.mlmonitor.langchain.xpuls_client import XpulsAILangChainClient 18 | from vishwa.mlmonitor.utils.common import find_key_in_nested_json 19 | 20 | 21 | def patch_invoke(ln_metrics: LangchainPrometheusMetrics, xpuls_client: XpulsAILangChainClient): 22 | # Store the original run method 23 | 24 | runnable_invoke = RunnableSequence.invoke 25 | runnable_ainvoke = RunnableSequence.ainvoke 26 | chain_invoke = Chain.invoke 27 | chain_ainvoke = Chain.ainvoke 28 | 29 | def _apply_patch(input: Input, config: Optional[RunnableConfig] = None, prompt_id: Optional[str] = None, 30 | prompt_version_id: Optional[str] = None): 31 | override_labels = get_scoped_override_labels() 32 | project_details = get_scoped_project_info() 33 | updated_labels = dict(ln_metrics.get_default_labels(), **override_labels) 34 | chain_run_id = str(uuid.uuid4()) 35 | 36 | ln_tracer = LangChainTracer( 37 | project_name=project_details['project_id'] if project_details['project_id'] is not None else 38 | project_details['project_slug'], 39 | client=xpuls_client, 40 | ) 41 | 42 | callback_handler = CallbackHandler(ln_metrics, chain_run_id, override_labels) 43 | 44 | updated_config = ensure_config(config) 45 | 46 | with ln_metrics.agent_run_histogram.labels(**dict(ln_metrics.get_default_labels(), **override_labels)).time(): 47 | if updated_config.get("callbacks") is not None: 48 | updated_config['callbacks'].append(callback_handler) 49 | else: 50 | updated_config['callbacks'] = [callback_handler] 51 | 52 | if vishwa.adv_tracing_enabled == "true": 53 | updated_config['callbacks'].append(ln_tracer) 54 | metadata = {'xpuls': {'labels': updated_labels, 'run_id': chain_run_id, 55 | 'project_id': project_details['project_id'] if project_details[ 56 | 'project_id'] is not None else 57 | project_details['project_slug'], 58 | "prompt_id": prompt_id, 59 | "prompt_version_id": prompt_version_id, 60 | }} 61 | 62 | updated_config['metadata'] = dict(updated_config['metadata'], **metadata) 63 | 64 | return updated_config, updated_labels 65 | 66 | def patched_chain_invoke(self, input: Dict[str, Any], 67 | config: Optional[RunnableConfig] = None, 68 | **kwargs: Any,): 69 | 70 | updated_config, updated_labels = _apply_patch(input=input, config=config) 71 | # Call the original run method 72 | return chain_invoke(self, input, updated_config, **kwargs) 73 | 74 | async def patched_chain_ainvoke(self, 75 | input: Dict[str, Any], 76 | config: Optional[RunnableConfig] = None, 77 | **kwargs: Any,): 78 | updated_config, updated_labels = _apply_patch(input=input, config=config) 79 | 80 | # Call the original run method 81 | return chain_ainvoke(self, input, updated_config, **kwargs) 82 | 83 | def patched_runnable_invoke(self, input: Input, config: Optional[RunnableConfig] = None): 84 | 85 | json_data = dumpd(self) 86 | prompt_id = find_key_in_nested_json(json_data, "prompt_id") 87 | prompt_version_id = find_key_in_nested_json(json_data, "prompt_version_id") 88 | updated_config, updated_labels = _apply_patch(input=input, config=config, prompt_id=prompt_id, 89 | prompt_version_id=prompt_version_id) 90 | # Call the original run method 91 | return runnable_invoke(self, input, updated_config) 92 | 93 | async def patched_runnable_ainvoke(self, input: Input, config: Optional[RunnableConfig] = None, **kwargs): 94 | json_data = dumpd(self) 95 | prompt_id = find_key_in_nested_json(json_data, "prompt_id") 96 | prompt_version_id = find_key_in_nested_json(json_data, "prompt_version_id") 97 | updated_config, updated_labels = _apply_patch(input=input, config=config, prompt_id=prompt_id, 98 | prompt_version_id=prompt_version_id) 99 | 100 | # Call the original run method 101 | return runnable_ainvoke(self, input, updated_config, **kwargs) 102 | 103 | # Patch the Chain class's invoke method with the new one 104 | Chain.invoke = patched_chain_invoke 105 | Chain.ainvoke = patched_chain_ainvoke 106 | 107 | # Patch the RunnableSequence class's invoke method with the new one 108 | RunnableSequence.invoke = patched_runnable_invoke 109 | RunnableSequence.ainvoke = patched_runnable_ainvoke 110 | -------------------------------------------------------------------------------- /vishwa/mlmonitor/langchain/patches/patch_run.py: -------------------------------------------------------------------------------- 1 | import os 2 | import uuid 3 | from typing import Dict, Any, Optional 4 | 5 | from langchain.callbacks import LangChainTracer 6 | from langchain.chains.base import Chain 7 | from langchain.schema.runnable import RunnableConfig, RunnableSequence 8 | from langsmith import Client 9 | 10 | import vishwa 11 | from vishwa.client.models import XPPrompt 12 | from vishwa.mlmonitor.langchain.decorators.telemetry_override_labels import TelemetryOverrideLabels 13 | from vishwa.mlmonitor.langchain.decorators.map_xpuls_project import MapXpulsProject 14 | from vishwa.mlmonitor.langchain.handlers.callback_handlers import CallbackHandler 15 | from vishwa.mlmonitor.langchain.patches.utils import get_scoped_project_info, get_scoped_override_labels 16 | 17 | from vishwa.mlmonitor.langchain.profiling.prometheus import LangchainPrometheusMetrics 18 | from vishwa.mlmonitor.langchain.xpuls_client import XpulsAILangChainClient 19 | 20 | 21 | def patch_run(ln_metrics: LangchainPrometheusMetrics, xpuls_client: XpulsAILangChainClient): 22 | # Store the original run method 23 | original_run = Chain.run 24 | original_arun = Chain.arun 25 | 26 | def _apply_patch(kwargs, prompt: Optional[XPPrompt] = None): 27 | override_labels = get_scoped_override_labels() 28 | project_details = get_scoped_project_info() 29 | 30 | updated_labels = dict(ln_metrics.get_default_labels(), **override_labels) 31 | chain_run_id = str(uuid.uuid4()) 32 | ln_tracer = LangChainTracer( 33 | project_name=project_details['project_id'] if project_details['project_id'] is not None else 34 | project_details['project_slug'], 35 | client=xpuls_client, 36 | ) 37 | 38 | callback_handler = CallbackHandler(ln_metrics, chain_run_id, override_labels) 39 | 40 | with ln_metrics.agent_run_histogram.labels(**updated_labels).time(): 41 | if 'callbacks' in kwargs: 42 | kwargs['callbacks'].append(callback_handler) 43 | else: 44 | kwargs['callbacks'] = [callback_handler] 45 | 46 | if vishwa.adv_tracing_enabled == "true": 47 | kwargs['callbacks'].append(ln_tracer) 48 | metadata = {'xpuls': {'labels': updated_labels, 'run_id': chain_run_id, 49 | 'prompt_id': prompt.prompt_id if prompt is not None else None, 50 | 'prompt_external_id': prompt.prompt_external_id if prompt is not None else None, 51 | 'prompt_version_id': prompt.prompt_version_id if prompt is not None else None, 52 | 'project_id': project_details['project_id'] if project_details[ 53 | 'project_id'] is not None else 54 | project_details['project_slug']}} 55 | if 'metadata' in kwargs: 56 | kwargs['metadata'] = dict(kwargs['metadata'], **metadata) 57 | else: 58 | kwargs['metadata'] = metadata 59 | return kwargs, ln_tracer, updated_labels 60 | 61 | def patched_run(self, *args, **kwargs): 62 | if args and not kwargs: 63 | if len(args) == 1 and isinstance(args[0], XPPrompt): 64 | updated_kwargs, ln_tracer, updated_labels = _apply_patch(kwargs, args[0]) 65 | prompt_text = args[0].prompt 66 | return original_run(self, prompt_text, **updated_kwargs) 67 | 68 | updated_kwargs, ln_tracer, updated_labels = _apply_patch(kwargs) 69 | 70 | # Call the original run method 71 | return original_run(self, *args, **updated_kwargs) 72 | 73 | async def patched_arun(self, *args, **kwargs): 74 | if args and not kwargs: 75 | if len(args) == 1 and isinstance(args[0], XPPrompt): 76 | updated_kwargs, ln_tracer, updated_labels = _apply_patch(kwargs, args[0]) 77 | prompt_text = args[0].prompt 78 | return original_arun(self, prompt_text, **updated_kwargs) 79 | 80 | updated_kwargs, ln_tracer, updated_labels = _apply_patch(kwargs) 81 | 82 | # Call the original run method 83 | return original_arun(self, *args, **updated_kwargs) 84 | 85 | # Patch the Chain class's run method with the new one 86 | Chain.run = patched_run 87 | Chain.arun = patched_arun 88 | 89 | 90 | -------------------------------------------------------------------------------- /vishwa/mlmonitor/langchain/patches/utils.py: -------------------------------------------------------------------------------- 1 | from vishwa.mlmonitor.langchain.decorators.map_xpuls_project import MapXpulsProject 2 | from vishwa.mlmonitor.langchain.decorators.telemetry_override_labels import TelemetryOverrideLabels 3 | 4 | 5 | def get_scoped_override_labels(): 6 | try: 7 | override_labels = TelemetryOverrideLabels._context.get() 8 | if override_labels is None: 9 | override_labels = {} 10 | except Exception as e: 11 | override_labels = {} 12 | return override_labels 13 | 14 | 15 | def get_scoped_project_info(): 16 | try: 17 | project_details = MapXpulsProject._context.get() 18 | if project_details is None: 19 | project_details = {'project_id': 'default'} 20 | except Exception as e: 21 | project_details = {'project_id': 'default'} 22 | return project_details 23 | -------------------------------------------------------------------------------- /vishwa/mlmonitor/langchain/patches/xp_prompt_template.py: -------------------------------------------------------------------------------- 1 | from abc import ABC 2 | from typing import Dict, Any, Optional 3 | 4 | import requests 5 | from langchain import BasePromptTemplate, PromptTemplate 6 | from langchain.prompts import ChatPromptTemplate, HumanMessagePromptTemplate 7 | from langchain.schema import PromptValue 8 | from langchain.schema.runnable import RunnableConfig 9 | 10 | from vishwa.client.models import XPPrompt 11 | 12 | 13 | class XPChatPromptTemplate(ChatPromptTemplate, ABC): 14 | @classmethod 15 | def from_template(cls, xp_prompt: XPPrompt, **kwargs: Any) -> ChatPromptTemplate: 16 | 17 | """ 18 | Overloaded method to create a custom chat prompt template from a template string. 19 | 20 | Args: 21 | xp_prompt: An XPPrompt instance. 22 | **kwargs: Additional keyword arguments. 23 | 24 | Returns: 25 | An instance of CustomChatPromptTemplate. 26 | """ 27 | # Perform custom actions or modifications here 28 | # For example, you might want to handle xp_prompt or kwargs differently 29 | 30 | kwargs["prompt_id"] = xp_prompt.prompt_id 31 | kwargs["prompt_external_id"] = xp_prompt.prompt_external_id 32 | kwargs["prompt_version_id"] = xp_prompt.prompt_version_id 33 | instance = super().from_template(xp_prompt.prompt, **kwargs) 34 | return instance 35 | -------------------------------------------------------------------------------- /vishwa/mlmonitor/langchain/profiling/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vishwa-labs/vishwa-ml-sdk/0491d3c5a1e2301643e1fc8e0f18cfaf503b91c7/vishwa/mlmonitor/langchain/profiling/__init__.py -------------------------------------------------------------------------------- /vishwa/mlmonitor/langchain/profiling/prometheus.py: -------------------------------------------------------------------------------- 1 | # Counters 2 | from typing import Dict, Any, Optional, List 3 | 4 | from prometheus_client import Counter, Histogram 5 | from pydantic import BaseModel 6 | 7 | chain_calls = Counter( 8 | 'langchain_chain_calls', 9 | 'Number of times method is called', 10 | ['type', 'lc', 'input_char_size', 'chat_history_char_size', 'run_id', 'parent_run_id', 11 | 'agent_type', 'model_type', 'model_name', 'other_tags']) 12 | chat_model_calls = Counter( 13 | 'langchain_chain_chat_model_calls', 14 | 'Counter for Langchain Chat Model Calls', 15 | ['type', 'lc', 'input_char_size', 'chat_history_char_size', 'run_id', 'parent_run_id', 16 | 'agent_type', 'deployment_name', 'model_name', 'temperature' 17 | 'other_tags']) 18 | llm_model_calls = Counter( 19 | 'langchain_chain_llm_model_calls', 20 | 'Counter for Langchain Chat Model Calls', 21 | ['type', 'lc', 'input_char_size', 'chat_history_char_size', 'run_id', 'parent_run_id', 22 | 'agent_type', 'deployment_name', 'model_name', 'temperature' 23 | 'other_tags']) 24 | 25 | #Histograms 26 | llm_latency = Histogram( 27 | 'langchain_chain_llm_latency', 28 | 'LLM latency', 29 | ['trace_id'] 30 | ) 31 | chain_latency = Histogram( 32 | 'chain_latency', 33 | 'Chain latency', 34 | ['trace_id'] 35 | ) 36 | 37 | class LangchainChainMetrics(BaseModel): 38 | lc: str 39 | type: str 40 | execution_step: str 41 | agent_type: str 42 | ml_model_type: Optional[str] 43 | ml_model_name: Optional[str] 44 | other_tags: Optional[str] 45 | # run_id: str 46 | # parent_run_id: Optional[str] 47 | # chain_run_id: str 48 | 49 | 50 | class LangchainToolMetrics(BaseModel): 51 | execution_step: str 52 | action: str 53 | agent_type: str 54 | other_tags: Optional[str] 55 | # run_id: str 56 | # parent_run_id: Optional[str] 57 | # chain_run_id: str 58 | 59 | 60 | class LLMTokens(BaseModel): 61 | execution_step: str 62 | ml_model_type: Optional[str] 63 | ml_model_name: Optional[str] 64 | other_tags: Optional[str] 65 | # run_id: str 66 | # parent_run_id: Optional[str] 67 | # chain_run_id: str 68 | 69 | 70 | class LangchainChatModelMetrics(BaseModel): 71 | lc: str 72 | type: str 73 | execution_step: str 74 | agent_type: str 75 | ml_model_type: Optional[str] 76 | ml_model_name: Optional[str] 77 | other_tags: Optional[str] 78 | # chain_run_id: str 79 | # run_id: str 80 | # parent_run_id: Optional[str] 81 | 82 | 83 | class LangchainPrometheusMetrics: 84 | def __init__(self, default_labels: Dict[str, Any]): 85 | chain_fields = list(LangchainChainMetrics.__fields__.keys()) + list(default_labels.keys()) 86 | chat_fields = list(LangchainChatModelMetrics.__fields__.keys()) + list(default_labels.keys()) 87 | llm_tokens_field = list(LLMTokens.__fields__.keys()) + list(default_labels.keys()) + ['usage_type'] 88 | tools_field = list(LangchainToolMetrics.__fields__.keys()) + list(default_labels.keys()) 89 | 90 | self.default_labels = default_labels 91 | self.chain_execution_counter = Counter( 92 | 'langchain_chain_execution', 93 | 'Langchain Chain Lifecycle counter', 94 | chain_fields 95 | ) 96 | self.chat_model_counter = Counter( 97 | 'langchain_chat_model', 98 | 'Langchain Chat Model counter', 99 | chat_fields 100 | ) 101 | self.llm_tokens_counter = Counter( 102 | 'langchain_llm_tokens', 103 | 'Langchain LLM Tokens Count', 104 | llm_tokens_field 105 | ) 106 | 107 | self.tools_usage_counter = Counter( 108 | 'langchain_tools_usage', 109 | 'Langchain tools Usage', 110 | tools_field 111 | ) 112 | self.chain_execution_histogram = Histogram( 113 | 'langchain_chain_latency', 114 | 'Langchain chain Latency', 115 | chain_fields 116 | ) 117 | 118 | self.chat_model_execution_histogram = Histogram( 119 | 'langchain_chat_model_latency', 120 | 'Langchain chat model Latency', 121 | chat_fields 122 | ) 123 | 124 | self.tools_execution_histogram = Histogram( 125 | 'langchain_tool_latency', 126 | 'Langchain tool Latency', 127 | tools_field 128 | ) 129 | 130 | ## Agent Run Latency 131 | self.agent_run_histogram = Histogram( 132 | 'langchain_agent_run_latency', 133 | 'Langchain Agent run end-to-end latency', 134 | list(default_labels.keys()) 135 | ) 136 | 137 | def get_default_labels(self): 138 | return self.default_labels 139 | 140 | def get_safe_override_labels(self, override_labels: Dict[str, str]): 141 | return {k: v for k, v in override_labels.items() if k in self.default_labels.keys()} 142 | 143 | def add_chain_counter(self, chain_metrics: LangchainChainMetrics, override_labels: Dict[str, str]): 144 | self.chain_execution_counter.labels( 145 | **dict(chain_metrics, **dict(self.default_labels, **self.get_safe_override_labels(override_labels))) 146 | ).inc() 147 | 148 | def add_tools_usage_counter(self, tool_metrics: LangchainToolMetrics, override_labels: Dict[str, str]): 149 | self.tools_usage_counter.labels( 150 | **dict(tool_metrics, **dict(self.default_labels, **self.get_safe_override_labels(override_labels))) 151 | ).inc() 152 | 153 | def add_llm_tokens_usage(self, openai_tokens: LLMTokens, 154 | usage_type: str, token_count: int, override_labels: Dict[str, str]): 155 | self.llm_tokens_counter.labels( 156 | **dict(openai_tokens, **{'usage_type': usage_type}, 157 | **dict(self.default_labels, **self.get_safe_override_labels(override_labels))) 158 | ).inc(token_count) 159 | 160 | def add_chat_model_counter(self, chat_metrics: LangchainChatModelMetrics, override_labels: Dict[str, str]): 161 | self.chat_model_counter.labels( 162 | **dict(chat_metrics, **dict(self.default_labels, **self.get_safe_override_labels(override_labels))) 163 | ).inc() 164 | 165 | def observe_chain_latency(self, chain_metrics: LangchainChainMetrics, elapsed_time: float, 166 | override_labels: Dict[str, str]): 167 | 168 | self.chain_execution_histogram.labels( 169 | **dict(chain_metrics, **dict(self.default_labels, **self.get_safe_override_labels(override_labels))) 170 | ).observe(elapsed_time) 171 | 172 | def observe_tool_latency(self, tool_metrics: LangchainToolMetrics, elapsed_time: float, 173 | override_labels: Dict[str, str]): 174 | 175 | self.tools_execution_histogram.labels( 176 | **dict(tool_metrics, **dict(self.default_labels, **self.get_safe_override_labels(override_labels))) 177 | ).observe(elapsed_time) 178 | 179 | def observe_chat_model_latency(self, model_metrics: LangchainChatModelMetrics, elapsed_time: float, 180 | override_labels: Dict[str, str]): 181 | 182 | self.chat_model_execution_histogram.labels( 183 | **dict(model_metrics, **dict(self.default_labels, **self.get_safe_override_labels(override_labels))) 184 | ).observe(elapsed_time) 185 | 186 | -------------------------------------------------------------------------------- /vishwa/mlmonitor/langchain/xpuls_client.py: -------------------------------------------------------------------------------- 1 | import functools 2 | import os 3 | import weakref 4 | from typing import Optional, Mapping, Dict 5 | 6 | import requests 7 | from langsmith import Client 8 | from langsmith import utils as ls_utils 9 | from urllib3 import Retry 10 | from requests import adapters as requests_adapters 11 | from urllib import parse as urllib_parse 12 | import socket 13 | 14 | import vishwa 15 | from vishwa.client import constants 16 | 17 | 18 | def _is_localhost(url: str) -> bool: 19 | """Check if the URL is localhost. 20 | 21 | Parameters 22 | ---------- 23 | url : str 24 | The URL to check. 25 | 26 | Returns 27 | ------- 28 | bool 29 | True if the URL is localhost, False otherwise. 30 | """ 31 | try: 32 | netloc = urllib_parse.urlsplit(url).netloc.split(":")[0] 33 | ip = socket.gethostbyname(netloc) 34 | return ip == "127.0.0.1" or ip.startswith("0.0.0.0") or ip.startswith("::") 35 | except socket.gaierror: 36 | return False 37 | 38 | 39 | def _get_api_key(api_key: Optional[str]) -> Optional[str]: 40 | api_key = api_key if api_key is not None else vishwa.api_key 41 | if api_key is None or not api_key.strip(): 42 | return None 43 | return api_key.strip().strip('"').strip("'") 44 | 45 | 46 | def _get_api_url(api_url: Optional[str], api_key: Optional[str]) -> str: 47 | _api_url = ( 48 | api_url 49 | if api_url is not None 50 | else vishwa.host_url 51 | ) 52 | if not _api_url.strip(): 53 | raise Exception("XpulsAI API URL cannot be empty") 54 | return _api_url.strip().strip('"').strip("'").rstrip("/") 55 | 56 | 57 | def _default_retry_config() -> Retry: 58 | """Get the default retry configuration. 59 | 60 | Returns 61 | ------- 62 | Retry 63 | The default retry configuration. 64 | """ 65 | return Retry( 66 | total=3, 67 | allowed_methods=None, # Retry on all methods 68 | status_forcelist=[502, 503, 504, 408, 425, 429], 69 | backoff_factor=0.5, 70 | # Sadly urllib3 1.x doesn't support backoff_jitter 71 | raise_on_redirect=False, 72 | raise_on_status=False, 73 | ) 74 | 75 | 76 | def close_session(session: requests.Session) -> None: 77 | """Close the session. 78 | 79 | Parameters 80 | ---------- 81 | session : Session 82 | The session to close. 83 | """ 84 | session.close() 85 | 86 | 87 | class XpulsAILangChainClient(Client): 88 | 89 | def __init__( 90 | self, 91 | api_url: Optional[str] = None, 92 | *, 93 | api_key: Optional[str] = None, 94 | retry_config: Optional[Retry] = None, 95 | timeout_ms: Optional[int] = None, 96 | ) -> None: 97 | self.api_key = _get_api_key(api_key) 98 | self.api_url = _get_api_url(api_url, self.api_key) 99 | self.retry_config = retry_config or _default_retry_config() 100 | self.timeout_ms = timeout_ms or 7000 101 | # Create a session and register a finalizer to close it 102 | self.session = requests.Session() 103 | weakref.finalize(self, close_session, self.session) 104 | 105 | # Mount the HTTPAdapter with the retry configuration 106 | adapter = requests_adapters.HTTPAdapter(max_retries=self.retry_config) 107 | self.session.mount("http://", adapter) 108 | self.session.mount("https://", adapter) 109 | self._get_data_type_cached = functools.lru_cache(maxsize=10)( 110 | self._get_data_type 111 | ) 112 | 113 | @property 114 | def _host_url(self) -> str: 115 | """The web host url.""" 116 | if _is_localhost(self.api_url): 117 | link = "http://localhost" 118 | else: 119 | link = self.api_url 120 | return link 121 | 122 | @property 123 | def _headers(self) -> Dict[str, str]: 124 | """Get the headers for the API request. 125 | 126 | Returns 127 | ------- 128 | Dict[str, str] 129 | The headers for the API request. 130 | """ 131 | headers = {} 132 | if self.api_key: 133 | headers["XP-API-Key"] = self.api_key 134 | return headers 135 | 136 | 137 | def request_with_retries( 138 | self, 139 | request_method: str, 140 | url: str, 141 | request_kwargs: Mapping, 142 | ) -> requests.Response: 143 | """Send a request with retries. 144 | 145 | Parameters 146 | ---------- 147 | request_method : str 148 | The HTTP request method. 149 | url : str 150 | The URL to send the request to. 151 | request_kwargs : Mapping 152 | Additional request parameters. 153 | 154 | Returns 155 | ------- 156 | Response 157 | The response object. 158 | 159 | """ 160 | try: 161 | # print(request_kwargs) 162 | response = self.session.request( 163 | request_method, url, stream=False, **request_kwargs 164 | ) 165 | ls_utils.raise_for_status_with_text(response) 166 | return response 167 | except requests.HTTPError as e: 168 | if response is not None and response.status_code == 500: 169 | raise Exception( 170 | f"Server error caused failure to {request_method} {url} in" 171 | f" XpulsAI API. {e}" 172 | ) 173 | else: 174 | raise Exception( 175 | f"Failed to {request_method} {url} in XpulsAI API. {e}" 176 | ) 177 | except requests.ConnectionError as e: 178 | raise Exception( 179 | f"Connection error caused failure to {request_method} {url}" 180 | " in XpulsAI API. Set environment variable `XPULSAI_HOST_URL`" 181 | f" {e}" 182 | ) from e 183 | except ValueError as e: 184 | args = list(e.args) 185 | msg = args[1] if len(args) > 1 else "" 186 | msg = msg.replace("session", "session (project)") 187 | emsg = "\n".join([args[0]] + [msg] + args[2:]) 188 | raise ls_utils.LangSmithError( 189 | f"Failed to {request_method} {url} in XpulsAI API. {emsg}" 190 | ) from e 191 | 192 | -------------------------------------------------------------------------------- /vishwa/mlmonitor/utils/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vishwa-labs/vishwa-ml-sdk/0491d3c5a1e2301643e1fc8e0f18cfaf503b91c7/vishwa/mlmonitor/utils/__init__.py -------------------------------------------------------------------------------- /vishwa/mlmonitor/utils/common.py: -------------------------------------------------------------------------------- 1 | from typing import Dict, Any 2 | 3 | 4 | def get_safe_dict_value(dictionary: Dict[Any, Any], key: str, default=None) -> Any: 5 | if key in dictionary: 6 | return dictionary[key] 7 | return default 8 | 9 | 10 | def find_key_in_nested_json(json_data, target_key): 11 | """ 12 | Recursively search for a key in a nested JSON structure. 13 | 14 | Args: 15 | json_data (dict): The JSON data to search through. 16 | target_key (str): The key to search for. 17 | 18 | Returns: 19 | The value of the found key, or None if the key is not found. 20 | """ 21 | if isinstance(json_data, dict): 22 | for key, value in json_data.items(): 23 | if key == target_key: 24 | return value 25 | else: 26 | result = find_key_in_nested_json(value, target_key) 27 | if result is not None: 28 | return result 29 | elif isinstance(json_data, list): 30 | for item in json_data: 31 | result = find_key_in_nested_json(item, target_key) 32 | if result is not None: 33 | return result 34 | return None 35 | -------------------------------------------------------------------------------- /vishwa/prompt_hub/__init__.py: -------------------------------------------------------------------------------- 1 | from .prompt import PromptClient 2 | -------------------------------------------------------------------------------- /vishwa/prompt_hub/prompt.py: -------------------------------------------------------------------------------- 1 | from typing import Dict 2 | import re 3 | 4 | from vishwa.client import XpulsAIClient 5 | from vishwa.client.models import XPPrompt 6 | 7 | 8 | class PromptClient: 9 | def __init__(self, prompt_id: str, environment_name: str): 10 | self._client = XpulsAIClient() 11 | self._prompt_id = prompt_id 12 | self._env_name = environment_name 13 | 14 | def get_prompt(self, variables: Dict[str, str]) -> XPPrompt: 15 | data = self._client.get_live_prompt( 16 | prompt_id=self._prompt_id, 17 | env_name=self._env_name 18 | ) 19 | """ 20 | Substitute variables in the prompt. 21 | 22 | Args: 23 | prompt (str): The prompt string with placeholders for variables. 24 | variables (Dict[str, str]): A dictionary of variable names and their values. 25 | 26 | Returns: 27 | str: The prompt with variables substituted. 28 | """ 29 | default_value = "" # Default value for variables not provided in the dictionary 30 | 31 | # Extracting variable names from the prompt 32 | prompt_variables = set(re.findall(r'\{\{(.+?)\}\}', data.prompt)) 33 | prompt = data.prompt 34 | for variable in prompt_variables: 35 | # Substitute the variable with its value or default value 36 | value = variables.get(variable, default_value) 37 | prompt = prompt.replace(f'{{{{{variable}}}}}', value) 38 | 39 | return XPPrompt( 40 | **{ 41 | "prompt_version_id": data.prompt_version_id, 42 | "prompt_id": data.prompt_id, 43 | "prompt_external_id": data.prompt_external_id, 44 | "prompt": prompt 45 | } 46 | ) 47 | --------------------------------------------------------------------------------