├── LICENSE ├── ModelMonitoring ├── ReadMe.md ├── Scrape_logs.py ├── config.ini ├── evaluate.py ├── ls_data.json ├── monitor_project_with_labelstudio.py ├── qalogs_11:12:24.txt ├── requirements.txt └── utils.py ├── README.md ├── beautiful_uis_workshop ├── labeling_config.txt └── process_data.ipynb ├── chatbot-prompter ├── Cornell Movies Example.ipynb ├── Customer Support Classification.ipynb ├── README.md └── prompter-workflow-screenshot.png ├── choose-best-llm ├── README.md ├── dashboard_server │ ├── get_stats.py │ ├── requirements.txt │ └── server.py ├── llms │ ├── Dockerfile │ ├── _wsgi.py │ ├── labeliing_schemes.py │ ├── llms.json │ ├── model.py │ ├── requirements-base.txt │ ├── requirements-test.txt │ └── requirements.txt ├── structured_output_config.xml └── tasks.json ├── fine-tuning ├── Fine_tune_Llama_3_8b_MedChat.ipynb ├── Medical-QA-Example.ipynb ├── README.md ├── requirements.txt └── tuned-llama3-8b ├── mistral-ocr ├── Labeling_Screen.png ├── MistralOCR-LabelStudio-example.ipynb └── ReadMe.md ├── multi-turn-chat ├── Evaluating Virtual Assistant Conversations.ipynb ├── Readme.md ├── labeled_results.json ├── multi-turn-chat-screenshot.png └── requirements.txt ├── openai-fine-tuning ├── OpenAI_fine-tuning.ipynb ├── README.md ├── annotated_qa_pairs.json ├── formatted_qa_list.json ├── qa_pairs_openai_wiki_hurricane_dataset_format.jsonl └── requirements.txt ├── paginated-multiimage-labeling ├── pdf2img.py └── pdfs │ └── rag.pdf ├── programmatic_actions_workshop └── SDK_notebook.ipynb ├── question-answering-system ├── README.md ├── add_documents.py ├── advanced_chatbot.py ├── build_doc_db.py ├── chatbot.py ├── images │ └── chat_ls_image.png ├── label_studio_callback_handler.py ├── requirements.txt └── simple_chatbot.py ├── rag-qa-review ├── RAG_QA_Review.ipynb ├── README.md ├── images │ └── rag_review.png └── requirements.txt ├── rlvr_math_tutor.ipynb ├── smol-docling ├── Labeling_Page.png ├── ReadMe.md └── SmolDocling-LabelStudio-example.ipynb └── structured-outputs └── OpenAI Structured Outputs with Label Studio.ipynb /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/LICENSE -------------------------------------------------------------------------------- /ModelMonitoring/ReadMe.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/ModelMonitoring/ReadMe.md -------------------------------------------------------------------------------- /ModelMonitoring/Scrape_logs.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/ModelMonitoring/Scrape_logs.py -------------------------------------------------------------------------------- /ModelMonitoring/config.ini: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/ModelMonitoring/config.ini -------------------------------------------------------------------------------- /ModelMonitoring/evaluate.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/ModelMonitoring/evaluate.py -------------------------------------------------------------------------------- /ModelMonitoring/ls_data.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/ModelMonitoring/ls_data.json -------------------------------------------------------------------------------- /ModelMonitoring/monitor_project_with_labelstudio.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/ModelMonitoring/monitor_project_with_labelstudio.py -------------------------------------------------------------------------------- /ModelMonitoring/qalogs_11:12:24.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/ModelMonitoring/qalogs_11:12:24.txt -------------------------------------------------------------------------------- /ModelMonitoring/requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/ModelMonitoring/requirements.txt -------------------------------------------------------------------------------- /ModelMonitoring/utils.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/ModelMonitoring/utils.py -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/README.md -------------------------------------------------------------------------------- /beautiful_uis_workshop/labeling_config.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/beautiful_uis_workshop/labeling_config.txt -------------------------------------------------------------------------------- /beautiful_uis_workshop/process_data.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/beautiful_uis_workshop/process_data.ipynb -------------------------------------------------------------------------------- /chatbot-prompter/Cornell Movies Example.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/chatbot-prompter/Cornell Movies Example.ipynb -------------------------------------------------------------------------------- /chatbot-prompter/Customer Support Classification.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/chatbot-prompter/Customer Support Classification.ipynb -------------------------------------------------------------------------------- /chatbot-prompter/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/chatbot-prompter/README.md -------------------------------------------------------------------------------- /chatbot-prompter/prompter-workflow-screenshot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/chatbot-prompter/prompter-workflow-screenshot.png -------------------------------------------------------------------------------- /choose-best-llm/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/choose-best-llm/README.md -------------------------------------------------------------------------------- /choose-best-llm/dashboard_server/get_stats.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/choose-best-llm/dashboard_server/get_stats.py -------------------------------------------------------------------------------- /choose-best-llm/dashboard_server/requirements.txt: -------------------------------------------------------------------------------- 1 | fastapi 2 | plotly -------------------------------------------------------------------------------- /choose-best-llm/dashboard_server/server.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/choose-best-llm/dashboard_server/server.py -------------------------------------------------------------------------------- /choose-best-llm/llms/Dockerfile: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/choose-best-llm/llms/Dockerfile -------------------------------------------------------------------------------- /choose-best-llm/llms/_wsgi.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/choose-best-llm/llms/_wsgi.py -------------------------------------------------------------------------------- /choose-best-llm/llms/labeliing_schemes.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/choose-best-llm/llms/labeliing_schemes.py -------------------------------------------------------------------------------- /choose-best-llm/llms/llms.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/choose-best-llm/llms/llms.json -------------------------------------------------------------------------------- /choose-best-llm/llms/model.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/choose-best-llm/llms/model.py -------------------------------------------------------------------------------- /choose-best-llm/llms/requirements-base.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/choose-best-llm/llms/requirements-base.txt -------------------------------------------------------------------------------- /choose-best-llm/llms/requirements-test.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/choose-best-llm/llms/requirements-test.txt -------------------------------------------------------------------------------- /choose-best-llm/llms/requirements.txt: -------------------------------------------------------------------------------- 1 | sglang[openai]==0.1.17 2 | outlines==0.0.45 3 | jsonschema==4.22.0 4 | -------------------------------------------------------------------------------- /choose-best-llm/structured_output_config.xml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/choose-best-llm/structured_output_config.xml -------------------------------------------------------------------------------- /choose-best-llm/tasks.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/choose-best-llm/tasks.json -------------------------------------------------------------------------------- /fine-tuning/Fine_tune_Llama_3_8b_MedChat.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/fine-tuning/Fine_tune_Llama_3_8b_MedChat.ipynb -------------------------------------------------------------------------------- /fine-tuning/Medical-QA-Example.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/fine-tuning/Medical-QA-Example.ipynb -------------------------------------------------------------------------------- /fine-tuning/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/fine-tuning/README.md -------------------------------------------------------------------------------- /fine-tuning/requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/fine-tuning/requirements.txt -------------------------------------------------------------------------------- /fine-tuning/tuned-llama3-8b: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/fine-tuning/tuned-llama3-8b -------------------------------------------------------------------------------- /mistral-ocr/Labeling_Screen.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/mistral-ocr/Labeling_Screen.png -------------------------------------------------------------------------------- /mistral-ocr/MistralOCR-LabelStudio-example.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/mistral-ocr/MistralOCR-LabelStudio-example.ipynb -------------------------------------------------------------------------------- /mistral-ocr/ReadMe.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/mistral-ocr/ReadMe.md -------------------------------------------------------------------------------- /multi-turn-chat/Evaluating Virtual Assistant Conversations.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/multi-turn-chat/Evaluating Virtual Assistant Conversations.ipynb -------------------------------------------------------------------------------- /multi-turn-chat/Readme.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/multi-turn-chat/Readme.md -------------------------------------------------------------------------------- /multi-turn-chat/labeled_results.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/multi-turn-chat/labeled_results.json -------------------------------------------------------------------------------- /multi-turn-chat/multi-turn-chat-screenshot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/multi-turn-chat/multi-turn-chat-screenshot.png -------------------------------------------------------------------------------- /multi-turn-chat/requirements.txt: -------------------------------------------------------------------------------- 1 | label-studio-sdk 2 | pandas 3 | -------------------------------------------------------------------------------- /openai-fine-tuning/OpenAI_fine-tuning.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/openai-fine-tuning/OpenAI_fine-tuning.ipynb -------------------------------------------------------------------------------- /openai-fine-tuning/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/openai-fine-tuning/README.md -------------------------------------------------------------------------------- /openai-fine-tuning/annotated_qa_pairs.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/openai-fine-tuning/annotated_qa_pairs.json -------------------------------------------------------------------------------- /openai-fine-tuning/formatted_qa_list.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/openai-fine-tuning/formatted_qa_list.json -------------------------------------------------------------------------------- /openai-fine-tuning/qa_pairs_openai_wiki_hurricane_dataset_format.jsonl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/openai-fine-tuning/qa_pairs_openai_wiki_hurricane_dataset_format.jsonl -------------------------------------------------------------------------------- /openai-fine-tuning/requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/openai-fine-tuning/requirements.txt -------------------------------------------------------------------------------- /paginated-multiimage-labeling/pdf2img.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/paginated-multiimage-labeling/pdf2img.py -------------------------------------------------------------------------------- /paginated-multiimage-labeling/pdfs/rag.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/paginated-multiimage-labeling/pdfs/rag.pdf -------------------------------------------------------------------------------- /programmatic_actions_workshop/SDK_notebook.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/programmatic_actions_workshop/SDK_notebook.ipynb -------------------------------------------------------------------------------- /question-answering-system/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/question-answering-system/README.md -------------------------------------------------------------------------------- /question-answering-system/add_documents.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/question-answering-system/add_documents.py -------------------------------------------------------------------------------- /question-answering-system/advanced_chatbot.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/question-answering-system/advanced_chatbot.py -------------------------------------------------------------------------------- /question-answering-system/build_doc_db.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/question-answering-system/build_doc_db.py -------------------------------------------------------------------------------- /question-answering-system/chatbot.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/question-answering-system/chatbot.py -------------------------------------------------------------------------------- /question-answering-system/images/chat_ls_image.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/question-answering-system/images/chat_ls_image.png -------------------------------------------------------------------------------- /question-answering-system/label_studio_callback_handler.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/question-answering-system/label_studio_callback_handler.py -------------------------------------------------------------------------------- /question-answering-system/requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/question-answering-system/requirements.txt -------------------------------------------------------------------------------- /question-answering-system/simple_chatbot.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/question-answering-system/simple_chatbot.py -------------------------------------------------------------------------------- /rag-qa-review/RAG_QA_Review.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/rag-qa-review/RAG_QA_Review.ipynb -------------------------------------------------------------------------------- /rag-qa-review/README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/rag-qa-review/README.md -------------------------------------------------------------------------------- /rag-qa-review/images/rag_review.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/rag-qa-review/images/rag_review.png -------------------------------------------------------------------------------- /rag-qa-review/requirements.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/rag-qa-review/requirements.txt -------------------------------------------------------------------------------- /rlvr_math_tutor.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/rlvr_math_tutor.ipynb -------------------------------------------------------------------------------- /smol-docling/Labeling_Page.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/smol-docling/Labeling_Page.png -------------------------------------------------------------------------------- /smol-docling/ReadMe.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/smol-docling/ReadMe.md -------------------------------------------------------------------------------- /smol-docling/SmolDocling-LabelStudio-example.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/smol-docling/SmolDocling-LabelStudio-example.ipynb -------------------------------------------------------------------------------- /structured-outputs/OpenAI Structured Outputs with Label Studio.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/HumanSignal/label-studio-examples/HEAD/structured-outputs/OpenAI Structured Outputs with Label Studio.ipynb --------------------------------------------------------------------------------