├── spec ├── dummy │ ├── log │ │ └── .keep │ ├── tmp │ │ ├── .keep │ │ ├── pids │ │ │ └── .keep │ │ ├── storage │ │ │ └── .keep │ │ └── local_secret.txt │ ├── storage │ │ └── .keep │ ├── app │ │ ├── assets │ │ │ ├── images │ │ │ │ └── .keep │ │ │ └── stylesheets │ │ │ │ └── application.css │ │ ├── models │ │ │ ├── concerns │ │ │ │ └── .keep │ │ │ ├── document.rb │ │ │ ├── application_record.rb │ │ │ └── raif │ │ │ │ ├── application_conversation.rb │ │ │ │ ├── test_user.rb │ │ │ │ └── conversations │ │ │ │ └── html_conversation_with_tools.rb │ │ ├── controllers │ │ │ ├── concerns │ │ │ │ └── .keep │ │ │ ├── application_controller.rb │ │ │ ├── chat_controller.rb │ │ │ └── agents_controller.rb │ │ ├── views │ │ │ ├── layouts │ │ │ │ ├── mailer.text.erb │ │ │ │ ├── mailer.html.erb │ │ │ │ └── application.html.erb │ │ │ ├── chat │ │ │ │ └── index.html.erb │ │ │ ├── agents │ │ │ │ ├── index.html.erb │ │ │ │ └── _conversation_history_entry.html.erb │ │ │ └── pwa │ │ │ │ ├── manifest.json.erb │ │ │ │ └── service-worker.js │ │ ├── helpers │ │ │ └── application_helper.rb │ │ ├── javascript │ │ │ ├── application.js │ │ │ └── controllers │ │ │ │ ├── application.js │ │ │ │ └── index.js │ │ └── jobs │ │ │ └── application_job.rb │ ├── raif_evals │ │ ├── results │ │ │ └── .gitignore │ │ └── setup.rb │ ├── public │ │ ├── icon.png │ │ └── icon.svg │ ├── bin │ │ ├── dev │ │ ├── importmap │ │ ├── rake │ │ ├── rails │ │ └── setup │ ├── config │ │ ├── environments │ │ │ └── development_mysql.rb │ │ ├── environment.rb │ │ ├── cable.yml │ │ ├── boot.rb │ │ ├── importmap.rb │ │ ├── initializers │ │ │ ├── assets.rb │ │ │ ├── filter_parameter_logging.rb │ │ │ ├── inflections.rb │ │ │ ├── raif.rb │ │ │ └── content_security_policy.rb │ │ ├── database.yml │ │ ├── routes.rb │ │ ├── locales │ │ │ └── en.yml │ │ ├── application.rb │ │ ├── storage.yml │ │ └── puma.rb │ ├── config.ru │ ├── db │ │ └── migrate │ │ │ ├── 20250225005128_create_users.rb │ │ │ └── 20250804015504_create_documents.rb │ └── Rakefile ├── fixtures │ ├── files │ │ ├── test.pdf │ │ └── cultivate.png │ └── llm_responses │ │ ├── anthropic │ │ └── developer_managed_fetch_url.json │ │ ├── open_router │ │ └── developer_managed_fetch_url.json │ │ └── open_ai_completions │ │ └── developer_managed_fetch_url.json ├── factories │ ├── test_users.rb │ └── shared │ │ ├── model_tool_invocations.rb │ │ ├── agents.rb │ │ ├── conversations.rb │ │ ├── conversation_entries.rb │ │ ├── tasks.rb │ │ └── model_completions.rb ├── models │ └── raif │ │ ├── model_tools │ │ ├── fetch_url_spec.rb │ │ ├── wikipedia_search_spec.rb │ │ └── agent_final_answer_spec.rb │ │ ├── embedding_model_spec.rb │ │ └── model_tool_spec.rb ├── support │ ├── test_conversation.rb │ ├── test_llm.rb │ ├── test_embedding_model.rb │ ├── test_model_tool.rb │ ├── current_temperature_test_tool.rb │ └── test_task.rb ├── setup │ └── capybara.rb ├── features │ └── raif │ │ └── admin │ │ └── config_spec.rb ├── i18n_spec.rb ├── jobs │ └── raif │ │ └── conversation_entry_job_spec.rb └── shared_examples │ └── agent.rb ├── bin ├── css ├── docs ├── clean_schema ├── rubocop ├── rails └── lint ├── .rspec ├── app ├── assets │ ├── config │ │ └── raif_manifest.js │ ├── stylesheets │ │ ├── raif.scss │ │ └── raif │ │ │ ├── conversations.scss │ │ │ └── admin │ │ │ ├── stats.scss │ │ │ └── conversation.scss │ └── javascript │ │ ├── raif │ │ ├── controllers │ │ │ └── conversations_controller.js │ │ └── stream_actions │ │ │ └── raif_scroll_to_bottom.js │ │ └── raif.js ├── views │ └── raif │ │ ├── conversations │ │ ├── show.html.erb │ │ ├── _initial_chat_message.html.erb │ │ ├── _conversation.html.erb │ │ ├── _available_user_tools.html.erb │ │ ├── _entry_processed.turbo_stream.erb │ │ ├── _full_conversation.html.erb │ │ └── index.html.erb │ │ ├── conversation_entries │ │ ├── _user_avatar.html.erb │ │ ├── _model_response_avatar.html.erb │ │ ├── new.turbo_stream.erb │ │ ├── _form_with_available_tools.html.erb │ │ ├── _citations.html.erb │ │ ├── create.turbo_stream.erb │ │ ├── _form_with_user_tool_invocation.html.erb │ │ ├── _message.html.erb │ │ ├── _form.html.erb │ │ └── _conversation_entry.html.erb │ │ └── admin │ │ ├── conversations │ │ ├── _conversation.html.erb │ │ └── index.html.erb │ │ ├── model_tools │ │ ├── _list.html.erb │ │ └── _model_tool.html.erb │ │ ├── tasks │ │ └── _task.html.erb │ │ ├── model_tool_invocations │ │ └── _model_tool_invocation.html.erb │ │ ├── model_completions │ │ ├── _model_completion.html.erb │ │ └── index.html.erb │ │ ├── agents │ │ ├── index.html.erb │ │ ├── _agent.html.erb │ │ └── _conversation_message.html.erb │ │ └── stats │ │ ├── model_tool_invocations │ │ └── index.html.erb │ │ └── _stats_tile.html.erb ├── models │ └── raif │ │ ├── model_image_input.rb │ │ ├── admin │ │ └── task_stat.rb │ │ ├── model_tools │ │ ├── provider_managed │ │ │ ├── base.rb │ │ │ ├── web_search.rb │ │ │ ├── code_execution.rb │ │ │ └── image_generation.rb │ │ ├── agent_final_answer.rb │ │ └── fetch_url.rb │ │ ├── concerns │ │ ├── invokes_model_tools.rb │ │ ├── has_available_model_tools.rb │ │ ├── llm_temperature.rb │ │ ├── has_llm.rb │ │ ├── has_requested_language.rb │ │ ├── llms │ │ │ ├── anthropic │ │ │ │ └── response_tool_calls.rb │ │ │ ├── open_ai_completions │ │ │ │ ├── response_tool_calls.rb │ │ │ │ └── tool_formatting.rb │ │ │ ├── open_ai_responses │ │ │ │ ├── response_tool_calls.rb │ │ │ │ └── tool_formatting.rb │ │ │ ├── bedrock │ │ │ │ ├── response_tool_calls.rb │ │ │ │ └── tool_formatting.rb │ │ │ └── message_formatting.rb │ │ └── agent_inference_stats.rb │ │ ├── embedding_model.rb │ │ ├── embedding_models │ │ ├── bedrock.rb │ │ └── open_ai.rb │ │ ├── application_record.rb │ │ ├── user_tool_invocation.rb │ │ └── streaming_responses │ │ └── open_ai_responses.rb ├── helpers │ └── raif │ │ ├── application_helper.rb │ │ └── shared │ │ └── conversations_helper.rb ├── jobs │ └── raif │ │ ├── application_job.rb │ │ └── conversation_entry_job.rb └── controllers │ └── raif │ ├── admin │ ├── agents_controller.rb │ ├── conversations_controller.rb │ ├── model_completions_controller.rb │ ├── stats │ │ ├── model_tool_invocations_controller.rb │ │ └── tasks_controller.rb │ ├── model_tool_invocations_controller.rb │ ├── tasks_controller.rb │ └── application_controller.rb │ └── application_controller.rb ├── docs ├── _includes │ ├── table-of-contents.md │ └── footer_custom.html ├── .gitignore ├── assets │ └── images │ │ ├── raif-logo-400.png │ │ └── screenshots │ │ ├── demo-app.png │ │ ├── admin-stats.png │ │ ├── admin-agents-show.png │ │ ├── admin-stats-tasks.png │ │ ├── admin-tasks-index.png │ │ ├── admin-tasks-show.png │ │ ├── admin-agents-index.png │ │ ├── conversation-interface.png │ │ ├── admin-conversation-show.png │ │ ├── admin-conversations-index.png │ │ ├── admin-model-completion-show.png │ │ ├── admin-model-completions-index.png │ │ ├── conversation-tool-invocation.png │ │ ├── admin-model-tool-invocation-show.png │ │ └── admin-model-tool-invocations-index.png ├── _sass │ ├── custom │ │ └── setup.scss │ └── color_schemes │ │ └── raif.scss ├── README.md ├── _learn_more │ ├── demo_app.md │ ├── embedding_models.md │ └── streaming.md └── Gemfile ├── lib ├── raif │ ├── version.rb │ ├── errors │ │ ├── invalid_config_error.rb │ │ ├── unsupported_feature_error.rb │ │ ├── action_not_authorized_error.rb │ │ ├── invalid_user_tool_type_error.rb │ │ ├── invalid_model_file_input_error.rb │ │ ├── invalid_model_image_input_error.rb │ │ ├── instance_dependent_schema_error.rb │ │ ├── invalid_conversation_type_error.rb │ │ ├── open_ai │ │ │ └── json_schema_error.rb │ │ └── streaming_error.rb │ ├── utils │ │ ├── html_to_markdown_converter.rb │ │ ├── colors.rb │ │ └── readable_content_extractor.rb │ ├── utils.rb │ ├── rspec.rb │ ├── languages.rb │ ├── errors.rb │ ├── evals.rb │ ├── evals │ │ ├── eval.rb │ │ ├── llm_judge.rb │ │ └── expectation_result.rb │ └── cli │ │ ├── evals_setup.rb │ │ ├── base.rb │ │ └── evals.rb ├── generators │ └── raif │ │ ├── task │ │ ├── templates │ │ │ ├── application_task.rb.tt │ │ │ └── task_eval_set.rb.tt │ │ └── task_generator.rb │ │ ├── agent │ │ ├── templates │ │ │ ├── application_agent.rb.tt │ │ │ ├── agent.rb.tt │ │ │ └── agent_eval_set.rb.tt │ │ └── agent_generator.rb │ │ ├── conversation │ │ └── templates │ │ │ └── application_conversation.rb.tt │ │ ├── base_generator.rb │ │ ├── eval_set │ │ ├── templates │ │ │ └── eval_set.rb.tt │ │ └── eval_set_generator.rb │ │ ├── model_tool │ │ ├── templates │ │ │ └── model_tool_invocation_partial.html.erb.tt │ │ └── model_tool_generator.rb │ │ ├── views_generator.rb │ │ ├── install │ │ └── install_generator.rb │ │ └── evals │ │ └── setup │ │ └── setup_generator.rb ├── tasks │ ├── raif_tasks.rake │ └── annotate_rb.rake └── raif.rb ├── exe └── raif ├── .gitignore ├── db └── migrate │ ├── 20250911125234_add_source_to_raif_tasks.rb │ ├── 20251020005853_add_source_to_raif_agents.rb │ ├── 20251020011346_rename_task_run_args_to_run_with.rb │ ├── 20251128202941_add_tool_choice_to_raif_model_completions.rb │ ├── 20250507155314_add_retry_count_to_raif_model_completions.rb │ ├── 20250421202149_add_response_format_to_raif_conversations.rb │ ├── 20250811171150_make_raif_task_creator_optional.rb │ ├── 20250603202013_add_stream_response_to_raif_model_completions.rb │ ├── 20251124185033_add_provider_tool_call_id_to_raif_model_tool_invocations.rb │ ├── 20250904194456_add_generating_entry_response_to_raif_conversations.rb │ ├── 20251020011405_add_run_with_to_raif_agents.rb │ ├── 20250804013843_add_task_run_args_to_raif_tasks.rb │ ├── 20250424232946_add_created_at_indexes.rb │ ├── 20250603140622_add_citations_to_raif_model_completions.rb │ ├── 20250527213016_add_response_id_and_response_array_to_model_completions.rb │ ├── 20251024160119_add_llm_messages_max_length_to_raif_conversations.rb │ ├── 20250502155330_add_status_indexes_to_raif_tasks.rb │ └── 20250424200755_add_cost_columns_to_raif_model_completions.rb ├── config ├── importmap.rb ├── initializers │ └── pagy.rb └── routes.rb ├── .github └── dependabot.yml ├── .claude ├── commands │ └── release-prep.md └── settings.local.json ├── Rakefile ├── gemfiles └── rails72.gemfile ├── Gemfile ├── .erb_lint.yml ├── MIT-LICENSE ├── CONTRIBUTING.md ├── package.json ├── .annotaterb.yml ├── vcr_cassettes ├── open_router │ └── text_response.yml ├── bedrock │ ├── json_response.yml │ └── text_response.yml └── open_ai_responses │ └── streaming_error.yml ├── raif.gemspec └── .rubocop.yml /spec/dummy/log/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /spec/dummy/tmp/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /spec/dummy/storage/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /spec/dummy/tmp/pids/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /spec/dummy/tmp/storage/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /spec/dummy/app/assets/images/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /spec/dummy/app/models/concerns/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /spec/dummy/app/controllers/concerns/.keep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /bin/css: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | bundle exec yarn watch:css -------------------------------------------------------------------------------- /.rspec: -------------------------------------------------------------------------------- 1 | --require spec_helper --format documentation 2 | -------------------------------------------------------------------------------- /spec/dummy/app/views/layouts/mailer.text.erb: -------------------------------------------------------------------------------- 1 | <%= yield %> 2 | -------------------------------------------------------------------------------- /spec/dummy/raif_evals/results/.gitignore: -------------------------------------------------------------------------------- 1 | * 2 | !.gitignore 3 | -------------------------------------------------------------------------------- /bin/docs: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | cd docs 4 | bundle exec jekyll serve -------------------------------------------------------------------------------- /bin/clean_schema: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | bundle exec rails app:db:migrate:reset -------------------------------------------------------------------------------- /app/assets/config/raif_manifest.js: -------------------------------------------------------------------------------- 1 | //= link_directory ../stylesheets/raif .css 2 | -------------------------------------------------------------------------------- /app/assets/stylesheets/raif.scss: -------------------------------------------------------------------------------- 1 | @use "raif/loader"; 2 | @use "raif/conversations"; -------------------------------------------------------------------------------- /app/views/raif/conversations/show.html.erb: -------------------------------------------------------------------------------- 1 | <%= raif_conversation(@conversation) %> 2 | -------------------------------------------------------------------------------- /spec/dummy/app/views/chat/index.html.erb: -------------------------------------------------------------------------------- 1 | <%= raif_conversation(@conversation) %> 2 | -------------------------------------------------------------------------------- /docs/_includes/table-of-contents.md: -------------------------------------------------------------------------------- 1 | ### Table of Contents 2 | {: .no_toc .text-delta } 3 | 1. TOC 4 | {:toc} -------------------------------------------------------------------------------- /spec/dummy/public/icon.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CultivateLabs/raif/HEAD/spec/dummy/public/icon.png -------------------------------------------------------------------------------- /lib/raif/version.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | VERSION = "1.4.0.pre" 5 | end 6 | -------------------------------------------------------------------------------- /spec/fixtures/files/test.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CultivateLabs/raif/HEAD/spec/fixtures/files/test.pdf -------------------------------------------------------------------------------- /app/assets/stylesheets/raif/conversations.scss: -------------------------------------------------------------------------------- 1 | .raif-conversation-entries-container { 2 | scroll-behavior: smooth; 3 | } -------------------------------------------------------------------------------- /docs/.gitignore: -------------------------------------------------------------------------------- 1 | _site/ 2 | .sass-cache/ 3 | .jekyll-cache/ 4 | .jekyll-metadata 5 | .bundle/ 6 | vendor/ 7 | Gemfile.lock -------------------------------------------------------------------------------- /spec/dummy/app/helpers/application_helper.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module ApplicationHelper 4 | end 5 | -------------------------------------------------------------------------------- /spec/dummy/app/models/document.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class Document < ApplicationRecord 4 | end 5 | -------------------------------------------------------------------------------- /spec/dummy/bin/dev: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | # frozen_string_literal: true 3 | 4 | exec "./bin/rails", "server", *ARGV 5 | -------------------------------------------------------------------------------- /spec/fixtures/files/cultivate.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CultivateLabs/raif/HEAD/spec/fixtures/files/cultivate.png -------------------------------------------------------------------------------- /docs/assets/images/raif-logo-400.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CultivateLabs/raif/HEAD/docs/assets/images/raif-logo-400.png -------------------------------------------------------------------------------- /app/views/raif/conversation_entries/_user_avatar.html.erb: -------------------------------------------------------------------------------- 1 | <%# so the host app can override to show a user avatar, if desired %> 2 | -------------------------------------------------------------------------------- /docs/_sass/custom/setup.scss: -------------------------------------------------------------------------------- 1 | // 2 | // Setup file to import color scheme variables 3 | // 4 | 5 | @import "./color_schemes/raif"; -------------------------------------------------------------------------------- /app/models/raif/model_image_input.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class Raif::ModelImageInput < Raif::ModelFileInput 4 | end 5 | -------------------------------------------------------------------------------- /app/views/raif/conversation_entries/_model_response_avatar.html.erb: -------------------------------------------------------------------------------- 1 | <%# so the host app can override to show a user avatar, if desired %> 2 | -------------------------------------------------------------------------------- /docs/assets/images/screenshots/demo-app.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CultivateLabs/raif/HEAD/docs/assets/images/screenshots/demo-app.png -------------------------------------------------------------------------------- /docs/assets/images/screenshots/admin-stats.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CultivateLabs/raif/HEAD/docs/assets/images/screenshots/admin-stats.png -------------------------------------------------------------------------------- /docs/assets/images/screenshots/admin-agents-show.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CultivateLabs/raif/HEAD/docs/assets/images/screenshots/admin-agents-show.png -------------------------------------------------------------------------------- /docs/assets/images/screenshots/admin-stats-tasks.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CultivateLabs/raif/HEAD/docs/assets/images/screenshots/admin-stats-tasks.png -------------------------------------------------------------------------------- /docs/assets/images/screenshots/admin-tasks-index.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CultivateLabs/raif/HEAD/docs/assets/images/screenshots/admin-tasks-index.png -------------------------------------------------------------------------------- /docs/assets/images/screenshots/admin-tasks-show.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CultivateLabs/raif/HEAD/docs/assets/images/screenshots/admin-tasks-show.png -------------------------------------------------------------------------------- /docs/_includes/footer_custom.html: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /docs/assets/images/screenshots/admin-agents-index.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CultivateLabs/raif/HEAD/docs/assets/images/screenshots/admin-agents-index.png -------------------------------------------------------------------------------- /spec/dummy/public/icon.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | -------------------------------------------------------------------------------- /spec/dummy/tmp/local_secret.txt: -------------------------------------------------------------------------------- 1 | 1e7ecad2ed971ddc81cab497d3a32562825efab2a6cfbfe699bb6c2d130835a01868e0c7b4a2473ce74dfc1eb421911abf922d5d4f5bcb83f95b66693a496da2 -------------------------------------------------------------------------------- /docs/assets/images/screenshots/conversation-interface.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CultivateLabs/raif/HEAD/docs/assets/images/screenshots/conversation-interface.png -------------------------------------------------------------------------------- /spec/dummy/app/models/application_record.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class ApplicationRecord < ActiveRecord::Base 4 | primary_abstract_class 5 | end 6 | -------------------------------------------------------------------------------- /spec/dummy/bin/importmap: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | # frozen_string_literal: true 3 | 4 | require_relative "../config/application" 5 | require "importmap/commands" 6 | -------------------------------------------------------------------------------- /spec/dummy/bin/rake: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | # frozen_string_literal: true 3 | 4 | require_relative "../config/boot" 5 | require "rake" 6 | Rake.application.run 7 | -------------------------------------------------------------------------------- /app/helpers/raif/application_helper.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module ApplicationHelper 5 | include Pagy::Frontend 6 | end 7 | end 8 | -------------------------------------------------------------------------------- /docs/assets/images/screenshots/admin-conversation-show.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CultivateLabs/raif/HEAD/docs/assets/images/screenshots/admin-conversation-show.png -------------------------------------------------------------------------------- /exe/raif: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | # frozen_string_literal: true 3 | 4 | require_relative "../lib/raif/cli" 5 | 6 | # Run the CLI 7 | Raif::CLI::Runner.new(ARGV).run 8 | -------------------------------------------------------------------------------- /spec/dummy/config/environments/development_mysql.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | require_relative "development" 4 | 5 | Rails.application.configure do 6 | end 7 | -------------------------------------------------------------------------------- /docs/assets/images/screenshots/admin-conversations-index.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CultivateLabs/raif/HEAD/docs/assets/images/screenshots/admin-conversations-index.png -------------------------------------------------------------------------------- /docs/assets/images/screenshots/admin-model-completion-show.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CultivateLabs/raif/HEAD/docs/assets/images/screenshots/admin-model-completion-show.png -------------------------------------------------------------------------------- /lib/generators/raif/task/templates/application_task.rb.tt: -------------------------------------------------------------------------------- 1 | module Raif 2 | class ApplicationTask < Raif::Task 3 | # Add any shared task behavior here 4 | end 5 | end 6 | -------------------------------------------------------------------------------- /docs/assets/images/screenshots/admin-model-completions-index.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CultivateLabs/raif/HEAD/docs/assets/images/screenshots/admin-model-completions-index.png -------------------------------------------------------------------------------- /docs/assets/images/screenshots/conversation-tool-invocation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CultivateLabs/raif/HEAD/docs/assets/images/screenshots/conversation-tool-invocation.png -------------------------------------------------------------------------------- /docs/assets/images/screenshots/admin-model-tool-invocation-show.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CultivateLabs/raif/HEAD/docs/assets/images/screenshots/admin-model-tool-invocation-show.png -------------------------------------------------------------------------------- /docs/assets/images/screenshots/admin-model-tool-invocations-index.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/CultivateLabs/raif/HEAD/docs/assets/images/screenshots/admin-model-tool-invocations-index.png -------------------------------------------------------------------------------- /lib/raif/errors/invalid_config_error.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Errors 5 | class InvalidConfigError < StandardError 6 | end 7 | end 8 | end 9 | -------------------------------------------------------------------------------- /app/jobs/raif/application_job.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | class ApplicationJob < ::ApplicationJob 5 | include ActionView::RecordIdentifier 6 | 7 | end 8 | end 9 | -------------------------------------------------------------------------------- /lib/raif/errors/unsupported_feature_error.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Errors 5 | class UnsupportedFeatureError < StandardError 6 | end 7 | end 8 | end 9 | -------------------------------------------------------------------------------- /lib/generators/raif/agent/templates/application_agent.rb.tt: -------------------------------------------------------------------------------- 1 | module Raif 2 | class ApplicationAgent < Raif::Agents::NativeToolCallingAgent 3 | # Add any shared agent behavior here 4 | end 5 | end 6 | -------------------------------------------------------------------------------- /lib/raif/errors/action_not_authorized_error.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Errors 5 | class ActionNotAuthorizedError < StandardError 6 | end 7 | end 8 | end 9 | -------------------------------------------------------------------------------- /lib/raif/errors/invalid_user_tool_type_error.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Errors 5 | class InvalidUserToolTypeError < StandardError 6 | end 7 | end 8 | end 9 | -------------------------------------------------------------------------------- /lib/raif/errors/invalid_model_file_input_error.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Errors 5 | class InvalidModelFileInputError < StandardError 6 | end 7 | end 8 | end 9 | -------------------------------------------------------------------------------- /lib/raif/errors/invalid_model_image_input_error.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Errors 5 | class InvalidModelImageInputError < StandardError 6 | end 7 | end 8 | end 9 | -------------------------------------------------------------------------------- /spec/dummy/raif_evals/setup.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | # 4 | # This file is loaded at the start of a run of your evals. 5 | # 6 | # Add any setup code that should run before your evals. 7 | # 8 | -------------------------------------------------------------------------------- /lib/generators/raif/conversation/templates/application_conversation.rb.tt: -------------------------------------------------------------------------------- 1 | module Raif 2 | class ApplicationConversation < Raif::Conversation 3 | # Add any shared conversation behavior here 4 | end 5 | end 6 | -------------------------------------------------------------------------------- /lib/raif/errors/instance_dependent_schema_error.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Errors 5 | class InstanceDependentSchemaError < StandardError 6 | end 7 | end 8 | end 9 | -------------------------------------------------------------------------------- /lib/raif/errors/invalid_conversation_type_error.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Errors 5 | class InvalidConversationTypeError < StandardError 6 | end 7 | end 8 | end 9 | -------------------------------------------------------------------------------- /spec/dummy/bin/rails: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | # frozen_string_literal: true 3 | 4 | APP_PATH = File.expand_path("../config/application", __dir__) 5 | require_relative "../config/boot" 6 | require "rails/commands" 7 | -------------------------------------------------------------------------------- /spec/dummy/config/environment.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | # Load the Rails application. 4 | require_relative "application" 5 | 6 | # Initialize the Rails application. 7 | Rails.application.initialize! 8 | -------------------------------------------------------------------------------- /app/models/raif/admin/task_stat.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Admin 5 | TaskStat = Data.define(:type, :llm_model_key, :count, :input_cost, :output_cost, :total_cost) 6 | end 7 | end 8 | -------------------------------------------------------------------------------- /spec/dummy/app/javascript/application.js: -------------------------------------------------------------------------------- 1 | // Configure your import map in config/importmap.rb. Read more: https://github.com/rails/importmap-rails 2 | import "@hotwired/turbo-rails" 3 | import "controllers" 4 | import "raif" 5 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | /.bundle/ 2 | /log/*.log 3 | /pkg/ 4 | /tmp/ 5 | /spec/dummy/log/*.log 6 | /spec/dummy/storage/ 7 | /spec/dummy/tmp/ 8 | /spec/examples.txt 9 | /node_modules/ 10 | /.yardoc/ 11 | /doc/ 12 | *.gem 13 | .erb_lint_cache/ -------------------------------------------------------------------------------- /lib/raif/utils/html_to_markdown_converter.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class Raif::Utils::HtmlToMarkdownConverter 4 | def self.convert(html) 5 | ReverseMarkdown.convert(html, unknown_tags: :bypass) 6 | end 7 | end 8 | -------------------------------------------------------------------------------- /spec/dummy/app/models/raif/application_conversation.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | class ApplicationConversation < Raif::Conversation 5 | # Add any shared conversation behavior here 6 | end 7 | end 8 | -------------------------------------------------------------------------------- /spec/dummy/config.ru: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | # This file is used by Rack-based servers to start the application. 4 | 5 | require_relative "config/environment" 6 | 7 | run Rails.application 8 | Rails.application.load_server 9 | -------------------------------------------------------------------------------- /app/views/raif/conversations/_initial_chat_message.html.erb: -------------------------------------------------------------------------------- 1 | <%# locals: (conversation:) %> 2 | 3 | <%= render "raif/conversation_entries/message", 4 | content: conversation.initial_chat_message, 5 | message_type: :model_response %> 6 | -------------------------------------------------------------------------------- /lib/raif/errors/open_ai/json_schema_error.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Errors 5 | module OpenAi 6 | class JsonSchemaError < StandardError 7 | end 8 | end 9 | end 10 | end 11 | -------------------------------------------------------------------------------- /app/assets/stylesheets/raif/admin/stats.scss: -------------------------------------------------------------------------------- 1 | .stats-icon { 2 | min-width: 46px; 3 | text-align: center; 4 | } 5 | 6 | .stats-card { 7 | transition: transform 0.2s; 8 | } 9 | 10 | .stats-card:hover { 11 | transform: translateY(-5px); 12 | } -------------------------------------------------------------------------------- /app/views/raif/conversation_entries/new.turbo_stream.erb: -------------------------------------------------------------------------------- 1 | <%= turbo_stream.update dom_id(@conversation, :entry_input) do %> 2 | <%= render @form_partial, 3 | conversation: @conversation, 4 | conversation_entry: @conversation_entry 5 | %> 6 | <% end %> -------------------------------------------------------------------------------- /app/models/raif/model_tools/provider_managed/base.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class Raif::ModelTools::ProviderManaged::Base < Raif::ModelTool 4 | class << self 5 | def provider_managed? 6 | true 7 | end 8 | end 9 | end 10 | -------------------------------------------------------------------------------- /db/migrate/20250911125234_add_source_to_raif_tasks.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class AddSourceToRaifTasks < ActiveRecord::Migration[7.1] 4 | def change 5 | add_reference :raif_tasks, :source, polymorphic: true, index: true 6 | end 7 | end 8 | -------------------------------------------------------------------------------- /db/migrate/20251020005853_add_source_to_raif_agents.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class AddSourceToRaifAgents < ActiveRecord::Migration[7.1] 4 | def change 5 | add_reference :raif_agents, :source, polymorphic: true, index: true 6 | end 7 | end 8 | -------------------------------------------------------------------------------- /db/migrate/20251020011346_rename_task_run_args_to_run_with.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class RenameTaskRunArgsToRunWith < ActiveRecord::Migration[7.1] 4 | def change 5 | rename_column :raif_tasks, :task_run_args, :run_with 6 | end 7 | end 8 | -------------------------------------------------------------------------------- /spec/dummy/config/cable.yml: -------------------------------------------------------------------------------- 1 | development: 2 | adapter: async 3 | 4 | test: 5 | adapter: test 6 | 7 | production: 8 | adapter: redis 9 | url: <%= ENV.fetch("REDIS_URL") { "redis://localhost:6379/1" } %> 10 | channel_prefix: dummy_production 11 | -------------------------------------------------------------------------------- /lib/raif/utils.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif::Utils 4 | require "raif/utils/readable_content_extractor" 5 | require "raif/utils/html_to_markdown_converter" 6 | require "raif/utils/html_fragment_processor" 7 | require "raif/utils/colors" 8 | end 9 | -------------------------------------------------------------------------------- /db/migrate/20251128202941_add_tool_choice_to_raif_model_completions.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class AddToolChoiceToRaifModelCompletions < ActiveRecord::Migration[7.2] 4 | def change 5 | add_column :raif_model_completions, :tool_choice, :string 6 | end 7 | end 8 | -------------------------------------------------------------------------------- /spec/dummy/app/models/raif/test_user.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class Raif::TestUser < ApplicationRecord 4 | has_one_attached :avatar 5 | has_many_attached :documents 6 | 7 | def preferred_language_key 8 | # no-op so we can stub in tests 9 | end 10 | end 11 | -------------------------------------------------------------------------------- /config/importmap.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | pin "raif", to: "raif.js" 4 | 5 | pin "raif/controllers/conversations_controller", to: "raif/controllers/conversations_controller.js" 6 | pin "raif/stream_actions/raif_scroll_to_bottom", to: "raif/stream_actions/raif_scroll_to_bottom.js" 7 | -------------------------------------------------------------------------------- /spec/dummy/db/migrate/20250225005128_create_users.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class CreateUsers < ActiveRecord::Migration[7.1] 4 | def change 5 | create_table :raif_test_users do |t| 6 | t.string :email 7 | 8 | t.timestamps 9 | end 10 | end 11 | end 12 | -------------------------------------------------------------------------------- /db/migrate/20250507155314_add_retry_count_to_raif_model_completions.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class AddRetryCountToRaifModelCompletions < ActiveRecord::Migration[7.1] 4 | def change 5 | add_column :raif_model_completions, :retry_count, :integer, default: 0, null: false 6 | end 7 | end 8 | -------------------------------------------------------------------------------- /spec/dummy/Rakefile: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | # Add your own tasks in files placed in lib/tasks ending in .rake, 4 | # for example lib/tasks/capistrano.rake, and they will automatically be available to Rake. 5 | 6 | require_relative "config/application" 7 | 8 | Rails.application.load_tasks 9 | -------------------------------------------------------------------------------- /db/migrate/20250421202149_add_response_format_to_raif_conversations.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class AddResponseFormatToRaifConversations < ActiveRecord::Migration[7.1] 4 | def change 5 | add_column :raif_conversations, :response_format, :integer, default: 0, null: false 6 | end 7 | end 8 | -------------------------------------------------------------------------------- /spec/dummy/app/javascript/controllers/application.js: -------------------------------------------------------------------------------- 1 | import { Application } from "@hotwired/stimulus" 2 | 3 | const application = Application.start() 4 | 5 | // Configure Stimulus development experience 6 | application.debug = false 7 | window.Stimulus = application 8 | 9 | export { application } 10 | -------------------------------------------------------------------------------- /db/migrate/20250811171150_make_raif_task_creator_optional.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class MakeRaifTaskCreatorOptional < ActiveRecord::Migration[7.1] 4 | def change 5 | change_column_null :raif_tasks, :creator_id, true 6 | change_column_null :raif_tasks, :creator_type, true 7 | end 8 | end 9 | -------------------------------------------------------------------------------- /.github/dependabot.yml: -------------------------------------------------------------------------------- 1 | version: 2 2 | updates: 3 | - package-ecosystem: bundler 4 | directory: "/" 5 | schedule: 6 | interval: daily 7 | open-pull-requests-limit: 10 8 | - package-ecosystem: github-actions 9 | directory: "/" 10 | schedule: 11 | interval: daily 12 | open-pull-requests-limit: 10 13 | -------------------------------------------------------------------------------- /app/views/raif/conversation_entries/_form_with_available_tools.html.erb: -------------------------------------------------------------------------------- 1 |
2 | <%= render "raif/conversations/available_user_tools", conversation: conversation %> 3 | <%= render "raif/conversation_entries/form", conversation: conversation, conversation_entry: conversation_entry %> 4 |
5 | -------------------------------------------------------------------------------- /spec/dummy/config/boot.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | # Set up gems listed in the Gemfile. 4 | ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../../../../Gemfile", __FILE__) 5 | 6 | require "bundler/setup" if File.exist?(ENV["BUNDLE_GEMFILE"]) 7 | $LOAD_PATH.unshift File.expand_path("../../../../lib", __FILE__) 8 | -------------------------------------------------------------------------------- /app/models/raif/model_tools/provider_managed/web_search.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class Raif::ModelTools::ProviderManaged::WebSearch < Raif::ModelTools::ProviderManaged::Base 4 | 5 | tool_description do 6 | "Utilizes the model provider's built-in web search capabilities." 7 | end 8 | 9 | end 10 | -------------------------------------------------------------------------------- /db/migrate/20250603202013_add_stream_response_to_raif_model_completions.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class AddStreamResponseToRaifModelCompletions < ActiveRecord::Migration[7.1] 4 | def change 5 | add_column :raif_model_completions, :stream_response, :boolean, default: false, null: false 6 | end 7 | end 8 | -------------------------------------------------------------------------------- /db/migrate/20251124185033_add_provider_tool_call_id_to_raif_model_tool_invocations.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class AddProviderToolCallIdToRaifModelToolInvocations < ActiveRecord::Migration[7.2] 4 | def change 5 | add_column :raif_model_tool_invocations, :provider_tool_call_id, :string 6 | end 7 | end 8 | -------------------------------------------------------------------------------- /lib/raif/rspec.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | require_relative "../../spec/support/rspec_helpers" 4 | require_relative "../../spec/support/test_model_tool" 5 | require_relative "../../spec/support/test_conversation" 6 | require_relative "../../spec/support/test_task" 7 | require_relative "../../spec/support/test_llm" 8 | -------------------------------------------------------------------------------- /spec/dummy/db/migrate/20250804015504_create_documents.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class CreateDocuments < ActiveRecord::Migration[7.1] 4 | def change 5 | create_table :documents do |t| 6 | t.string :title 7 | t.text :content 8 | 9 | t.timestamps 10 | end 11 | end 12 | end 13 | -------------------------------------------------------------------------------- /lib/tasks/raif_tasks.rake: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | namespace :raif do 4 | namespace :install do 5 | desc "Copy migrations from Raif to host application" 6 | task :migrations do 7 | ENV["FROM"] = "raif" 8 | Rake::Task["railties:install:migrations"].invoke 9 | end 10 | end 11 | end 12 | -------------------------------------------------------------------------------- /spec/dummy/app/javascript/controllers/index.js: -------------------------------------------------------------------------------- 1 | // Import and register all your controllers from the importmap via controllers/**/*_controller 2 | import { application } from "controllers/application" 3 | import { eagerLoadControllersFrom } from "@hotwired/stimulus-loading" 4 | eagerLoadControllersFrom("controllers", application) 5 | -------------------------------------------------------------------------------- /spec/dummy/app/views/layouts/mailer.html.erb: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 8 | 9 | 10 | 11 | <%= yield %> 12 | 13 | 14 | -------------------------------------------------------------------------------- /spec/dummy/app/views/agents/index.html.erb: -------------------------------------------------------------------------------- 1 | <%= turbo_stream_from :agents %> 2 | 3 | <%= form_with url: agents_path, method: :post do |form| %> 4 | <%= form.text_field :task, placeholder: "Enter a task", class: "form-control" %> 5 | <%= form.submit "Run Agent" %> 6 | <% end %> 7 | 8 |
9 |
10 | -------------------------------------------------------------------------------- /app/models/raif/model_tools/provider_managed/code_execution.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class Raif::ModelTools::ProviderManaged::CodeExecution < Raif::ModelTools::ProviderManaged::Base 4 | 5 | tool_description do 6 | "Utilizes the model provider's built-in code execution capabilities." 7 | end 8 | 9 | end 10 | -------------------------------------------------------------------------------- /db/migrate/20250904194456_add_generating_entry_response_to_raif_conversations.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class AddGeneratingEntryResponseToRaifConversations < ActiveRecord::Migration[7.1] 4 | def change 5 | add_column :raif_conversations, :generating_entry_response, :boolean, default: false, null: false 6 | end 7 | end 8 | -------------------------------------------------------------------------------- /app/models/raif/model_tools/provider_managed/image_generation.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class Raif::ModelTools::ProviderManaged::ImageGeneration < Raif::ModelTools::ProviderManaged::Base 4 | 5 | tool_description do 6 | "Utilizes the model provider's built-in image generation capabilities." 7 | end 8 | 9 | end 10 | -------------------------------------------------------------------------------- /app/helpers/raif/shared/conversations_helper.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Shared 5 | module ConversationsHelper 6 | 7 | def raif_conversation(conversation) 8 | render "raif/conversations/full_conversation", conversation: conversation 9 | end 10 | 11 | end 12 | end 13 | end 14 | -------------------------------------------------------------------------------- /lib/tasks/annotate_rb.rake: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | # This rake task was added by annotate_rb gem. 4 | 5 | # Can set `ANNOTATERB_SKIP_ON_DB_TASKS` to be anything to skip this 6 | if Rails.env.development? && ENV["ANNOTATERB_SKIP_ON_DB_TASKS"].nil? 7 | require "annotate_rb" 8 | 9 | AnnotateRb::Core.load_rake_tasks 10 | end 11 | -------------------------------------------------------------------------------- /bin/rubocop: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | # frozen_string_literal: true 3 | 4 | require "rubygems" 5 | require "bundler/setup" 6 | 7 | # explicit rubocop config increases performance slightly while avoiding config confusion. 8 | ARGV.unshift("--config", File.expand_path("../.rubocop.yml", __dir__)) 9 | 10 | load Gem.bin_path("rubocop", "rubocop") 11 | -------------------------------------------------------------------------------- /app/assets/javascript/raif/controllers/conversations_controller.js: -------------------------------------------------------------------------------- 1 | import { Controller } from "@hotwired/stimulus"; 2 | 3 | export default class extends Controller { 4 | connect() { 5 | requestAnimationFrame(() => this.scrollToBottom()); 6 | } 7 | 8 | scrollToBottom() { 9 | this.element.scrollTo({ top: this.element.scrollHeight }); 10 | } 11 | } -------------------------------------------------------------------------------- /app/models/raif/concerns/invokes_model_tools.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif::Concerns::InvokesModelTools 4 | extend ActiveSupport::Concern 5 | 6 | included do 7 | has_many :raif_model_tool_invocations, 8 | class_name: "Raif::ModelToolInvocation", 9 | as: :source, 10 | dependent: :destroy 11 | end 12 | 13 | end 14 | -------------------------------------------------------------------------------- /app/views/raif/conversations/_conversation.html.erb: -------------------------------------------------------------------------------- 1 | <%# locals: (conversation:) %> 2 | 3 | 4 | <%= conversation.created_at.strftime("%B %d, %Y") %> 5 | 6 | <%= link_to t("raif.conversations.index.table.view"), 7 | raif.conversation_path(conversation), 8 | class: "btn btn-sm btn-primary" %> 9 | 10 | 11 | -------------------------------------------------------------------------------- /spec/dummy/app/jobs/application_job.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class ApplicationJob < ActiveJob::Base 4 | # Automatically retry jobs that encountered a deadlock 5 | # retry_on ActiveRecord::Deadlocked 6 | 7 | # Most jobs are safe to ignore if the underlying records are no longer available 8 | # discard_on ActiveJob::DeserializationError 9 | end 10 | -------------------------------------------------------------------------------- /spec/dummy/config/importmap.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | # Pin npm packages by running ./bin/importmap 4 | 5 | pin "application" 6 | pin "@hotwired/turbo-rails", to: "turbo.js" 7 | pin "@hotwired/stimulus", to: "stimulus.js" 8 | pin "@hotwired/stimulus-loading", to: "stimulus-loading.js" 9 | pin_all_from "app/javascript/controllers", under: "controllers" 10 | -------------------------------------------------------------------------------- /app/assets/javascript/raif.js: -------------------------------------------------------------------------------- 1 | // Register all Raif controllers 2 | import { application } from "controllers/application" 3 | 4 | import ConversationsController from "raif/controllers/conversations_controller" 5 | application.register("raif--conversations", ConversationsController) 6 | 7 | export { ConversationsController } 8 | 9 | import "raif/stream_actions/raif_scroll_to_bottom" 10 | 11 | -------------------------------------------------------------------------------- /db/migrate/20251020011405_add_run_with_to_raif_agents.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class AddRunWithToRaifAgents < ActiveRecord::Migration[7.1] 4 | def change 5 | json_column_type = if connection.adapter_name.downcase.include?("postgresql") 6 | :jsonb 7 | else 8 | :json 9 | end 10 | 11 | add_column :raif_agents, :run_with, json_column_type 12 | end 13 | end 14 | -------------------------------------------------------------------------------- /spec/factories/test_users.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | FactoryBot.define do 4 | factory :raif_test_user, class: "Raif::TestUser" do 5 | sequence(:email){|i| "user-#{SecureRandom.hex(3)}-#{i}@example.com" } 6 | end 7 | 8 | trait :with_avatar do 9 | avatar { Rack::Test::UploadedFile.new(Raif::Engine.root.join("spec/fixtures/files/cultivate.png"), "image/png") } 10 | end 11 | end 12 | -------------------------------------------------------------------------------- /db/migrate/20250804013843_add_task_run_args_to_raif_tasks.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class AddTaskRunArgsToRaifTasks < ActiveRecord::Migration[7.1] 4 | def change 5 | json_column_type = if connection.adapter_name.downcase.include?("postgresql") 6 | :jsonb 7 | else 8 | :json 9 | end 10 | 11 | add_column :raif_tasks, :task_run_args, json_column_type 12 | end 13 | end 14 | -------------------------------------------------------------------------------- /.claude/commands/release-prep.md: -------------------------------------------------------------------------------- 1 | Please prep Raif for a release. To do this: 2 | 3 | - Create a branch called `version/v-release-prep` 4 | - Remove `(Unreleased)` from CHANGELOG.md 5 | - Remove `-pre` from version file 6 | - Run `bundle install` 7 | - Commit the changes 8 | - Push the branch to `origin` 9 | - Run `gh pr create --base main --title "Prepare for release v" --body "Release v"` -------------------------------------------------------------------------------- /spec/models/raif/model_tools/fetch_url_spec.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | require "rails_helper" 4 | 5 | RSpec.describe Raif::ModelTools::FetchUrl do 6 | describe "#tool_arguments_schema" do 7 | it "validates against OpenAI's rules" do 8 | llm = Raif.llm(:open_ai_gpt_4o_mini) 9 | expect(llm.validate_json_schema!(described_class.tool_arguments_schema)).to eq(true) 10 | end 11 | end 12 | end 13 | -------------------------------------------------------------------------------- /app/models/raif/concerns/has_available_model_tools.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif::Concerns::HasAvailableModelTools 4 | extend ActiveSupport::Concern 5 | 6 | def available_model_tools_map 7 | available_model_tools&.map do |tool_name| 8 | tool_klass = tool_name.is_a?(String) ? tool_name.constantize : tool_name 9 | [tool_klass.tool_name, tool_klass] 10 | end.to_h 11 | end 12 | 13 | end 14 | -------------------------------------------------------------------------------- /db/migrate/20250424232946_add_created_at_indexes.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class AddCreatedAtIndexes < ActiveRecord::Migration[7.1] 4 | def change 5 | add_index :raif_model_completions, :created_at 6 | add_index :raif_tasks, :created_at 7 | add_index :raif_conversations, :created_at 8 | add_index :raif_conversation_entries, :created_at 9 | add_index :raif_agents, :created_at 10 | end 11 | end 12 | -------------------------------------------------------------------------------- /db/migrate/20250603140622_add_citations_to_raif_model_completions.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class AddCitationsToRaifModelCompletions < ActiveRecord::Migration[7.1] 4 | def change 5 | json_column_type = if connection.adapter_name.downcase.include?("postgresql") 6 | :jsonb 7 | else 8 | :json 9 | end 10 | 11 | add_column :raif_model_completions, :citations, json_column_type 12 | end 13 | end 14 | -------------------------------------------------------------------------------- /spec/models/raif/model_tools/wikipedia_search_spec.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | require "rails_helper" 4 | 5 | RSpec.describe Raif::ModelTools::WikipediaSearch do 6 | describe "#tool_arguments_schema" do 7 | it "validates against OpenAI's rules" do 8 | llm = Raif.llm(:open_ai_gpt_4o_mini) 9 | expect(llm.validate_json_schema!(described_class.tool_arguments_schema)).to eq(true) 10 | end 11 | end 12 | end 13 | -------------------------------------------------------------------------------- /spec/models/raif/model_tools/agent_final_answer_spec.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | require "rails_helper" 4 | 5 | RSpec.describe Raif::ModelTools::AgentFinalAnswer do 6 | describe "#tool_arguments_schema" do 7 | it "validates against OpenAI's rules" do 8 | llm = Raif.llm(:open_ai_gpt_4o_mini) 9 | expect(llm.validate_json_schema!(described_class.tool_arguments_schema)).to eq(true) 10 | end 11 | end 12 | end 13 | -------------------------------------------------------------------------------- /app/controllers/raif/admin/agents_controller.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Admin 5 | class AgentsController < Raif::Admin::ApplicationController 6 | include Pagy::Backend 7 | 8 | def index 9 | @pagy, @agents = pagy(Raif::Agent.order(created_at: :desc)) 10 | end 11 | 12 | def show 13 | @agent = Raif::Agent.find(params[:id]) 14 | end 15 | end 16 | end 17 | end 18 | -------------------------------------------------------------------------------- /spec/dummy/app/controllers/application_controller.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class ApplicationController < ActionController::Base 4 | # Only allow modern browsers supporting webp images, web push, badges, import maps, CSS nesting, and CSS :has. 5 | allow_browser versions: :modern 6 | 7 | # Used by Raif to get the current user 8 | def current_user 9 | @current_user ||= Raif::TestUser.find_or_create_by(email: "test@example.com") 10 | end 11 | end 12 | -------------------------------------------------------------------------------- /spec/dummy/config/initializers/assets.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | # Be sure to restart your server when you modify this file. 4 | 5 | # Rails 8 uses propshaft for the asset pipeline 6 | 7 | # Add additional assets to the asset load path if needed 8 | # Rails.application.config.assets.paths << Rails.root.join("node_modules") 9 | 10 | # Set an explicit assets prefix if needed (defaults to "/assets") 11 | # Rails.application.config.assets.prefix = "/assets" 12 | -------------------------------------------------------------------------------- /spec/factories/shared/model_tool_invocations.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | FactoryBot.define do 4 | factory :raif_model_tool_invocation, class: "Raif::ModelToolInvocation" do 5 | source { create(:raif_conversation_entry) } 6 | tool_type { "Raif::TestModelTool" } 7 | tool_arguments { { "items": [{ "title": "foo", "description": "bar" }] } } 8 | 9 | trait :with_result do 10 | result { { "status": "success" } } 11 | end 12 | end 13 | end 14 | -------------------------------------------------------------------------------- /lib/raif/utils/colors.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Utils 5 | module Colors 6 | def self.green(text) 7 | "\e[32m#{text}\e[0m" 8 | end 9 | 10 | def self.red(text) 11 | "\e[31m#{text}\e[0m" 12 | end 13 | 14 | def self.yellow(text) 15 | "\e[33m#{text}\e[0m" 16 | end 17 | 18 | def self.blue(text) 19 | "\e[34m#{text}\e[0m" 20 | end 21 | end 22 | end 23 | end 24 | -------------------------------------------------------------------------------- /app/controllers/raif/admin/conversations_controller.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Admin 5 | class ConversationsController < Raif::Admin::ApplicationController 6 | include Pagy::Backend 7 | 8 | def index 9 | @pagy, @conversations = pagy(Raif::Conversation.order(created_at: :desc)) 10 | end 11 | 12 | def show 13 | @conversation = Raif::Conversation.find(params[:id]) 14 | end 15 | end 16 | end 17 | end 18 | -------------------------------------------------------------------------------- /spec/factories/shared/agents.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | FactoryBot.define do 4 | factory :raif_agent, class: "Raif::Agent" do 5 | task { "What is Jimmy Buffet's birthday?" } 6 | available_model_tools { ["Raif::ModelTools::WikipediaSearch", "Raif::ModelTools::FetchUrl"] } 7 | creator { FB.create(:raif_test_user) } 8 | end 9 | 10 | factory :raif_native_tool_calling_agent, parent: :raif_agent, class: "Raif::Agents::NativeToolCallingAgent" do 11 | end 12 | end 13 | -------------------------------------------------------------------------------- /spec/support/test_conversation.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class Raif::TestConversation < Raif::Conversation 4 | 5 | before_create :populate_available_model_tools 6 | 7 | def populate_available_model_tools 8 | self.available_model_tools = [ 9 | "Raif::TestModelTool", 10 | "Raif::ModelTools::WikipediaSearch", 11 | ] 12 | end 13 | 14 | def process_model_response_message(message:, entry:) 15 | message&.gsub("jerk", "[REDACTED]") 16 | end 17 | 18 | end 19 | -------------------------------------------------------------------------------- /app/controllers/raif/admin/model_completions_controller.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Admin 5 | class ModelCompletionsController < Raif::Admin::ApplicationController 6 | include Pagy::Backend 7 | 8 | def index 9 | @pagy, @model_completions = pagy(Raif::ModelCompletion.order(created_at: :desc)) 10 | end 11 | 12 | def show 13 | @model_completion = Raif::ModelCompletion.find(params[:id]) 14 | end 15 | end 16 | end 17 | end 18 | -------------------------------------------------------------------------------- /Rakefile: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | require "bundler/setup" 4 | 5 | APP_RAKEFILE = File.expand_path("spec/dummy/Rakefile", __dir__) 6 | load "rails/tasks/engine.rake" 7 | 8 | load "rails/tasks/statistics.rake" 9 | 10 | require "bundler/gem_tasks" 11 | 12 | begin 13 | require "yard" 14 | YARD::Rake::YardocTask.new do |t| 15 | t.files = ["lib/**/*.rb", "app/**/*.rb", "-", "README.md"] 16 | t.options = ["--output-dir=doc"] 17 | end 18 | rescue LoadError 19 | # YARD not available 20 | end 21 | -------------------------------------------------------------------------------- /app/views/raif/conversation_entries/_citations.html.erb: -------------------------------------------------------------------------------- 1 |
2 |
<%= t("raif.common.sources") %>
3 | <% conversation_entry.citations.each do |citation| %> 4 | <%= link_to citation["url"], target: "_blank", rel: "noopener noreferrer", class: "d-flex align-items-center" do %> 5 | <%= image_tag "https://www.google.com/s2/favicons?sz=16&domain=#{citation["url"]}", class: "me-1" %> 6 | <%= citation["title"] %> 7 | <% end %> 8 | <% end %> 9 |
10 | -------------------------------------------------------------------------------- /lib/raif/errors/streaming_error.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Errors 5 | class StreamingError < StandardError 6 | attr_reader :type, :code, :event 7 | 8 | def initialize(message:, type:, event:, code: nil) 9 | super(message) 10 | 11 | @type = type 12 | @code = code 13 | @event = event 14 | end 15 | 16 | def to_s 17 | "[#{type}] #{super} (code=#{code}, event=#{event})" 18 | end 19 | end 20 | end 21 | end 22 | -------------------------------------------------------------------------------- /spec/dummy/app/views/pwa/manifest.json.erb: -------------------------------------------------------------------------------- 1 | { 2 | "name": "Dummy", 3 | "icons": [ 4 | { 5 | "src": "/icon.png", 6 | "type": "image/png", 7 | "sizes": "512x512" 8 | }, 9 | { 10 | "src": "/icon.png", 11 | "type": "image/png", 12 | "sizes": "512x512", 13 | "purpose": "maskable" 14 | } 15 | ], 16 | "start_url": "/", 17 | "display": "standalone", 18 | "scope": "/", 19 | "description": "Dummy.", 20 | "theme_color": "red", 21 | "background_color": "red" 22 | } 23 | -------------------------------------------------------------------------------- /app/views/raif/admin/conversations/_conversation.html.erb: -------------------------------------------------------------------------------- 1 | 2 | <%= link_to "##{conversation.id}", raif.admin_conversation_path(conversation) %> 3 | <%= conversation.created_at.rfc822 %> 4 | <%= conversation.creator.try(:raif_display_name) || "#{conversation.creator_type} ##{conversation.creator_id}" %> 5 | <%= conversation.type %> 6 | <%= conversation.conversation_entries_count %> 7 | 8 | -------------------------------------------------------------------------------- /db/migrate/20250527213016_add_response_id_and_response_array_to_model_completions.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class AddResponseIdAndResponseArrayToModelCompletions < ActiveRecord::Migration[7.1] 4 | def change 5 | json_column_type = if connection.adapter_name.downcase.include?("postgresql") 6 | :jsonb 7 | else 8 | :json 9 | end 10 | 11 | add_column :raif_model_completions, :response_id, :string 12 | add_column :raif_model_completions, :response_array, json_column_type 13 | end 14 | end 15 | -------------------------------------------------------------------------------- /app/assets/javascript/raif/stream_actions/raif_scroll_to_bottom.js: -------------------------------------------------------------------------------- 1 | import { Turbo } from "@hotwired/turbo-rails"; 2 | 3 | Turbo.StreamActions.raif_scroll_to_bottom = function () { 4 | const targetSelector = this.getAttribute("target"); 5 | const targetElement = document.getElementById(targetSelector); 6 | 7 | if (targetElement) { 8 | targetElement.scrollTo({ top: targetElement.scrollHeight, behavior: "smooth" }); 9 | } else { 10 | console.warn(`scrollToBottom: No element found for selector '${targetSelector}'`); 11 | } 12 | }; -------------------------------------------------------------------------------- /app/models/raif/concerns/llm_temperature.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif::Concerns::LlmTemperature 4 | extend ActiveSupport::Concern 5 | 6 | included do 7 | class_attribute :temperature, instance_writer: false 8 | end 9 | 10 | class_methods do 11 | def llm_temperature(temperature) 12 | raise ArgumentError, "temperature must be a number between 0 and 1" unless temperature.is_a?(Numeric) && temperature.between?(0, 1) 13 | 14 | self.temperature = temperature 15 | end 16 | end 17 | end 18 | -------------------------------------------------------------------------------- /lib/raif/languages.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | SUPPORTED_LANGUAGES = [ 5 | "ar", 6 | "da", 7 | "de", 8 | "en", 9 | "es", 10 | "fi", 11 | "fr", 12 | "he", 13 | "hi", 14 | "it", 15 | "ja", 16 | "ko", 17 | "nl", 18 | "no", 19 | "pl", 20 | "pt", 21 | "ru", 22 | "sv", 23 | "th", 24 | "tr", 25 | "uk", 26 | "vi", 27 | "zh", 28 | ].freeze 29 | 30 | def self.supported_languages 31 | SUPPORTED_LANGUAGES 32 | end 33 | end 34 | -------------------------------------------------------------------------------- /spec/dummy/config/database.yml: -------------------------------------------------------------------------------- 1 | default: &default 2 | adapter: postgresql 3 | encoding: unicode 4 | pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %> 5 | 6 | development: 7 | <<: *default 8 | database: raif_dummy_development 9 | 10 | test: 11 | <<: *default 12 | database: raif_dummy_test 13 | 14 | development_mysql: &mysql 15 | adapter: mysql2 16 | encoding: utf8mb4 17 | pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %> 18 | username: root 19 | password: 20 | host: localhost 21 | database: raif_dummy_development_mysql -------------------------------------------------------------------------------- /app/models/raif/concerns/has_llm.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif::Concerns::HasLlm 4 | extend ActiveSupport::Concern 5 | 6 | included do 7 | validates :llm_model_key, presence: true, inclusion: { in: ->{ Raif.available_llm_keys.map(&:to_s) } } 8 | 9 | before_validation ->{ self.llm_model_key ||= default_llm_model_key } 10 | end 11 | 12 | def default_llm_model_key 13 | Raif.config.default_llm_model_key 14 | end 15 | 16 | def llm 17 | @llm ||= Raif.llm(llm_model_key.to_sym) 18 | end 19 | end 20 | -------------------------------------------------------------------------------- /docs/README.md: -------------------------------------------------------------------------------- 1 | # Raif Documentation 2 | 3 | This directory contains the Jekyll-based documentation site for Raif, built with the [just-the-docs](https://just-the-docs.github.io/just-the-docs/) theme. 4 | 5 | ## Setup 6 | 7 | 1. Navigate to the docs directory: 8 | ```bash 9 | cd docs 10 | ``` 11 | 12 | 2. Install dependencies: 13 | ```bash 14 | bundle install 15 | ``` 16 | 17 | 3. Serve the site locally: 18 | ```bash 19 | bundle exec jekyll serve 20 | ``` 21 | 22 | 4. Open your browser to `http://127.0.0.1:4000/raif/` 23 | -------------------------------------------------------------------------------- /app/views/raif/conversation_entries/create.turbo_stream.erb: -------------------------------------------------------------------------------- 1 | <% if @conversation_entry.persisted? %> 2 | <%= turbo_stream.append dom_id(@conversation, :entries), @conversation_entry %> 3 | <%= turbo_stream.action :raif_scroll_to_bottom, dom_id(@conversation, :entries) %> 4 | 5 | <%= turbo_stream.update dom_id(@conversation, :entry_input) do %> 6 | <%= render "raif/conversation_entries/form_with_available_tools", 7 | conversation: @conversation, 8 | conversation_entry: Raif::ConversationEntry.new 9 | %> 10 | <% end %> 11 | <% end %> -------------------------------------------------------------------------------- /db/migrate/20251024160119_add_llm_messages_max_length_to_raif_conversations.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class AddLlmMessagesMaxLengthToRaifConversations < ActiveRecord::Migration[7.1] 4 | def change 5 | add_column :raif_conversations, :llm_messages_max_length, :integer 6 | 7 | reversible do |dir| 8 | dir.up do 9 | # Set default value for existing conversations 10 | execute "UPDATE raif_conversations SET llm_messages_max_length = 50 WHERE llm_messages_max_length IS NULL" 11 | end 12 | end 13 | end 14 | end 15 | -------------------------------------------------------------------------------- /gemfiles/rails72.gemfile: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | source "https://rubygems.org" 4 | 5 | gemspec path: ".." 6 | 7 | gem "rails", "~> 7.2.0" 8 | 9 | gem "puma" 10 | gem "pg" 11 | gem "mysql2" 12 | gem "guard-rspec", require: false 13 | gem "factory_bot_rails" 14 | gem "debug", platforms: [:mri] 15 | gem "rspec-rails" 16 | gem "rubocop-shopify" 17 | gem "i18n-tasks" 18 | gem "erb_lint" 19 | gem "capybara" 20 | gem "propshaft" 21 | gem "importmap-rails" 22 | gem "stimulus-rails" 23 | gem "cuprite" 24 | gem "webmock" 25 | gem "yard" 26 | gem "vcr" 27 | -------------------------------------------------------------------------------- /config/initializers/pagy.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | # Pagy initializer file 4 | # See https://ddnexus.github.io/pagy/api/pagy#backend 5 | 6 | # Optionally override some pagy default options 7 | # Pagy::DEFAULT[:items] = 20 # items per page 8 | # Pagy::DEFAULT[:size] = [1,4,4,1] # nav bar links 9 | 10 | # When you are done setting your own default freeze it, so it will not get changed accidentally 11 | # Pagy::DEFAULT.freeze 12 | 13 | # Add the pagy backend to a controller or to an object that includes it 14 | require "pagy/extras/bootstrap" 15 | -------------------------------------------------------------------------------- /db/migrate/20250502155330_add_status_indexes_to_raif_tasks.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class AddStatusIndexesToRaifTasks < ActiveRecord::Migration[7.1] 4 | def change 5 | add_index :raif_tasks, :completed_at 6 | add_index :raif_tasks, :failed_at 7 | add_index :raif_tasks, :started_at 8 | 9 | # Index for type + status combinations which will be common in the admin interface 10 | add_index :raif_tasks, [:type, :completed_at] 11 | add_index :raif_tasks, [:type, :failed_at] 12 | add_index :raif_tasks, [:type, :started_at] 13 | end 14 | end 15 | -------------------------------------------------------------------------------- /lib/raif/errors.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | require "raif/errors/invalid_config_error" 4 | require "raif/errors/action_not_authorized_error" 5 | require "raif/errors/invalid_user_tool_type_error" 6 | require "raif/errors/invalid_conversation_type_error" 7 | require "raif/errors/open_ai/json_schema_error" 8 | require "raif/errors/invalid_model_image_input_error" 9 | require "raif/errors/invalid_model_file_input_error" 10 | require "raif/errors/unsupported_feature_error" 11 | require "raif/errors/streaming_error" 12 | require "raif/errors/instance_dependent_schema_error" 13 | -------------------------------------------------------------------------------- /spec/dummy/config/initializers/filter_parameter_logging.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | # Be sure to restart your server when you modify this file. 4 | 5 | # Configure parameters to be partially matched (e.g. passw matches password) and filtered from the log file. 6 | # Use this to limit dissemination of sensitive information. 7 | # See the ActiveSupport::ParameterFilter documentation for supported notations and behaviors. 8 | Rails.application.config.filter_parameters += [ 9 | :passw, :email, :secret, :token, :_key, :crypt, :salt, :certificate, :otp, :ssn, :cvv, :cvc 10 | ] 11 | -------------------------------------------------------------------------------- /lib/generators/raif/base_generator.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | class BaseGenerator < Rails::Generators::NamedBase 5 | private 6 | 7 | def raif_module_namespacing(intermediate_modules = [], &block) 8 | content = capture(&block).rstrip 9 | 10 | modules_names = intermediate_modules + class_path.map(&:camelize) 11 | modules_names.reverse.each do |module_name| 12 | content = indent "module #{module_name}\n#{content}\nend", 2 13 | end 14 | 15 | concat("module Raif\n#{content}\nend\n") 16 | end 17 | 18 | end 19 | end 20 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | source "https://rubygems.org" 4 | 5 | # Specify your gem's dependencies in raif.gemspec. 6 | gemspec 7 | 8 | gem "puma" 9 | gem "pg" 10 | gem "mysql2" 11 | gem "guard-rspec", require: false 12 | gem "factory_bot_rails" 13 | gem "debug", platforms: [:mri] 14 | gem "rspec-rails" 15 | gem "rubocop-shopify" 16 | gem "i18n-tasks" 17 | gem "erb_lint" 18 | gem "capybara" 19 | gem "propshaft" 20 | gem "importmap-rails" 21 | gem "stimulus-rails" 22 | gem "cuprite" 23 | gem "webmock" 24 | gem "vcr" 25 | gem "yard" 26 | gem "annotaterb" 27 | gem "openssl", "3.3.2" 28 | -------------------------------------------------------------------------------- /spec/dummy/app/controllers/chat_controller.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class ChatController < ApplicationController 4 | def index 5 | conversation_type = params[:conversation_type] == "html" ? Raif::Conversations::HtmlConversationWithTools : Raif::Conversation 6 | # Find the latest conversation for this user or create a new one 7 | @conversation = conversation_type.where(creator: current_user).newest_first.first 8 | 9 | if @conversation.nil? 10 | @conversation = conversation_type.new(creator: current_user) 11 | @conversation.save! 12 | end 13 | end 14 | end 15 | -------------------------------------------------------------------------------- /spec/factories/shared/conversations.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | FactoryBot.define do 4 | factory :raif_conversation, class: "Raif::Conversation" do 5 | trait :with_entries do 6 | transient do 7 | entries_count { 3 } 8 | end 9 | 10 | after(:create) do |conversation, evaluator| 11 | create_list(:raif_conversation_entry, evaluator.entries_count, :completed, raif_conversation: conversation) 12 | end 13 | end 14 | end 15 | 16 | factory :raif_test_conversation, class: "Raif::TestConversation", parent: :raif_conversation do 17 | end 18 | end 19 | -------------------------------------------------------------------------------- /spec/setup/capybara.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | require "capybara/cuprite" 4 | 5 | Capybara.javascript_driver = :cuprite 6 | Capybara.register_driver(:cuprite) do |app| 7 | headless = ENV["HEADLESS"] != "false" 8 | browser_options = { "no-sandbox": nil } 9 | 10 | opts = { 11 | browser_options: browser_options, 12 | flatten: false, 13 | process_timeout: 25, 14 | window_size: [1440, 900], 15 | headless: headless, 16 | } 17 | 18 | opts[:slowmo] = 0.01 unless headless 19 | Capybara::Cuprite::Driver.new(app, opts) 20 | end 21 | 22 | Capybara.disable_animation = true 23 | -------------------------------------------------------------------------------- /app/views/raif/conversations/_available_user_tools.html.erb: -------------------------------------------------------------------------------- 1 | <% if conversation.available_user_tools.any? %> 2 |

<%= t("raif.common.tools") %>

3 |
4 | <% conversation.available_user_tool_classes.each do |tool| %> 5 | <%= link_to tool.tool_name, 6 | raif.new_conversation_entry_path(conversation, user_tool_type: tool.name), 7 | class: "btn btn-sm btn-phoenix-secondary rounded-pill fs-10 me-1 mb-1", 8 | data: { turbo_stream: true } %> 9 | <% end %> 10 |
11 | <% end %> 12 | -------------------------------------------------------------------------------- /app/controllers/raif/application_controller.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | class ApplicationController < ::ApplicationController 5 | 6 | before_action :authorize_raif_action 7 | 8 | def raif_current_user 9 | send(Raif.config.current_user_method) if respond_to?(Raif.config.current_user_method) 10 | end 11 | 12 | private 13 | 14 | def authorize_raif_action 15 | unless instance_exec(&Raif.config.authorize_controller_action) 16 | raise Raif::Errors::ActionNotAuthorizedError, "#{self.class.name}##{action_name} not authorized" 17 | end 18 | end 19 | end 20 | end 21 | -------------------------------------------------------------------------------- /bin/rails: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | # frozen_string_literal: true 3 | 4 | # This command will automatically be run when you run "rails" with Rails gems 5 | # installed from the root of your application. 6 | 7 | ENGINE_ROOT = File.expand_path("..", __dir__) 8 | ENGINE_PATH = File.expand_path("../lib/raif/engine", __dir__) 9 | APP_PATH = File.expand_path("../spec/dummy/config/application", __dir__) 10 | 11 | # Set up gems listed in the Gemfile. 12 | ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../Gemfile", __dir__) 13 | require "bundler/setup" if File.exist?(ENV["BUNDLE_GEMFILE"]) 14 | 15 | require "rails/all" 16 | require "rails/engine/commands" 17 | -------------------------------------------------------------------------------- /app/views/raif/conversations/_entry_processed.turbo_stream.erb: -------------------------------------------------------------------------------- 1 | <%# locals: (conversation:, conversation_entry:) %> 2 | 3 | <%# Update the conversation entry now that it's been processed & has a model response %> 4 | <%= turbo_stream.replace conversation_entry do %> 5 | <%= render conversation_entry %> 6 | <% end %> 7 | 8 | <%= turbo_stream.replace dom_id(conversation, "entry_form") do %> 9 | <%= render "raif/conversation_entries/form", conversation: conversation, conversation_entry: Raif::ConversationEntry.new %> 10 | <% end %> 11 | 12 | <%= turbo_stream.action :raif_scroll_to_bottom, ActionView::RecordIdentifier.dom_id(conversation, :entries) %> -------------------------------------------------------------------------------- /lib/generators/raif/eval_set/templates/eval_set.rb.tt: -------------------------------------------------------------------------------- 1 | <% raif_module_namespacing(["Evals"]) do -%> 2 | class <%= class_name.demodulize %>EvalSet < Raif::Evals::EvalSet 3 | # Run this eval set with: 4 | # bundle exec raif evals ./<%= eval_set_file_path %> 5 | 6 | # Setup method runs before each eval 7 | setup do 8 | # Common setup code 9 | end 10 | 11 | # Teardown runs after each eval 12 | teardown do 13 | # Cleanup code 14 | end 15 | 16 | eval "description of your eval" do 17 | # Your eval code here 18 | # expect_tool_invocation, expect_no_tool_invocation, expect, etc. 19 | end 20 | end 21 | <% end -%> -------------------------------------------------------------------------------- /lib/generators/raif/model_tool/templates/model_tool_invocation_partial.html.erb.tt: -------------------------------------------------------------------------------- 1 | <%%# 2 | This partial is used to render a model tool invocation to the user in the conversation interface. 3 | If you don't want the tool invocation to be displayed to the user, you can override the `renderable?` method in your model tool class to return false 4 | %> 5 | 6 |
7 |
<%%= <%= file_name %>.tool_type.demodulize.titleize %> Result
8 | 
<%%= JSON.pretty_generate(<%= file_name %>.result || {}) %>
9 |

Edit this file in <%%= __FILE__ %> to customize the display of the tool invocation.

10 |
-------------------------------------------------------------------------------- /lib/raif/evals.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | require "raif/evals/expectation_result" 4 | require "raif/evals/eval" 5 | require "raif/evals/eval_set" 6 | require "raif/evals/run" 7 | require "raif/evals/llm_judge" 8 | require "raif/evals/llm_judges/binary" 9 | require "raif/evals/llm_judges/comparative" 10 | require "raif/evals/llm_judges/scored" 11 | require "raif/evals/llm_judges/summarization" 12 | require "raif/evals/scoring_rubric" 13 | 14 | module Raif 15 | module Evals 16 | # Namespace modules for organizing eval sets 17 | module Tasks 18 | end 19 | 20 | module Conversations 21 | end 22 | 23 | module Agents 24 | end 25 | end 26 | end 27 | -------------------------------------------------------------------------------- /app/views/raif/admin/model_tools/_list.html.erb: -------------------------------------------------------------------------------- 1 |
2 |
3 |
<%= t("raif.admin.common.available_tools") %>
4 |
5 |
6 | <% if model_tools.blank? %> 7 | <%= t("raif.admin.common.none") %> 8 | <% else %> 9 |
10 | <% model_tools.each_with_index do |(tool_name, tool_class), index| %> 11 | <%= render partial: "raif/admin/model_tools/model_tool", locals: { tool_name: tool_name, tool_class: tool_class, index: index } %> 12 | <% end %> 13 |
14 | <% end %> 15 |
16 |
17 | -------------------------------------------------------------------------------- /spec/dummy/app/views/agents/_conversation_history_entry.html.erb: -------------------------------------------------------------------------------- 1 |
2 | <% if conversation_history_entry["type"] == "tool_call" %> 3 | <%= t("dummy.tool_call") %>: <%= conversation_history_entry["name"] %>(<%= conversation_history_entry["arguments"].to_json %>) 4 | <% elsif conversation_history_entry["type"] == "tool_call_result" %> 5 | <%= t("dummy.tool_result") %>: <%= conversation_history_entry["result"].is_a?(String) ? conversation_history_entry["result"].truncate(200) : conversation_history_entry["result"].to_json.truncate(200) %> 6 | <% else %> 7 | <%= conversation_history_entry["role"] %>: <%= conversation_history_entry["content"] %> 8 | <% end %> 9 |
10 | -------------------------------------------------------------------------------- /app/models/raif/embedding_model.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class Raif::EmbeddingModel 4 | include ActiveModel::Model 5 | 6 | attr_accessor :key, 7 | :api_name, 8 | :input_token_cost, 9 | :default_output_vector_size 10 | 11 | validates :default_output_vector_size, presence: true, numericality: { only_integer: true, greater_than: 0 } 12 | validates :api_name, presence: true 13 | validates :key, presence: true 14 | 15 | def name 16 | I18n.t("raif.embedding_model_names.#{key}") 17 | end 18 | 19 | def generate_embedding!(input, dimensions: nil) 20 | raise NotImplementedError, "#{self.class.name} must implement #generate_embedding!" 21 | end 22 | end 23 | -------------------------------------------------------------------------------- /app/views/raif/conversations/_full_conversation.html.erb: -------------------------------------------------------------------------------- 1 | <%= turbo_stream_from conversation %> 2 | 3 |
4 | <%= render conversation.initial_chat_message_partial_path, conversation: conversation %> 5 | <%= render conversation.entries.oldest_first %> 6 |
7 | 8 |
9 | <%= render "raif/conversation_entries/form_with_available_tools", 10 | conversation: conversation, 11 | conversation_entry: Raif::ConversationEntry.new %> 12 |
13 | -------------------------------------------------------------------------------- /spec/factories/shared/conversation_entries.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | FactoryBot.define do 4 | factory :raif_conversation_entry, class: "Raif::ConversationEntry" do 5 | sequence(:user_message){|i| "User message #{i} #{SecureRandom.hex(4)}" } 6 | creator { raif_conversation.creator } 7 | 8 | trait :completed do 9 | sequence(:model_response_message){|i| "Model response #{i} #{SecureRandom.hex(4)}" } 10 | started_at { Time.current } 11 | completed_at { Time.current } 12 | end 13 | 14 | trait :with_tool_invocation do 15 | after(:create) do |entry| 16 | create(:raif_model_tool_invocation, source: entry) 17 | end 18 | end 19 | end 20 | end 21 | -------------------------------------------------------------------------------- /app/models/raif/concerns/has_requested_language.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif::Concerns::HasRequestedLanguage 4 | extend ActiveSupport::Concern 5 | 6 | included do 7 | validates :requested_language_key, inclusion: { in: Raif.supported_languages, allow_blank: true } 8 | end 9 | 10 | def requested_language_name 11 | @requested_language_name ||= I18n.t("raif.languages.#{requested_language_key}", locale: "en") 12 | end 13 | 14 | def system_prompt_language_preference 15 | return if requested_language_key.blank? 16 | 17 | "\nYou're collaborating with teammate who speaks #{requested_language_name}. Please respond in #{requested_language_name}." 18 | end 19 | 20 | end 21 | -------------------------------------------------------------------------------- /app/models/raif/concerns/llms/anthropic/response_tool_calls.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif::Concerns::Llms::Anthropic::ResponseToolCalls 4 | extend ActiveSupport::Concern 5 | 6 | def extract_response_tool_calls(resp) 7 | return if resp&.dig("content").nil? 8 | 9 | # Find any tool_use content blocks 10 | tool_uses = resp&.dig("content")&.select do |content| 11 | content["type"] == "tool_use" 12 | end 13 | 14 | return if tool_uses.blank? 15 | 16 | tool_uses.map do |tool_use| 17 | { 18 | "provider_tool_call_id" => tool_use["id"], 19 | "name" => tool_use["name"], 20 | "arguments" => tool_use["input"], 21 | } 22 | end 23 | end 24 | end 25 | -------------------------------------------------------------------------------- /db/migrate/20250424200755_add_cost_columns_to_raif_model_completions.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class AddCostColumnsToRaifModelCompletions < ActiveRecord::Migration[7.1] 4 | # If you need to backfill cost columns for existing records: 5 | # Raif::ModelCompletion.find_each do |model_completion| 6 | # model_completion.calculate_costs 7 | # model_completion.save(validate: false) 8 | # end 9 | def change 10 | add_column :raif_model_completions, :prompt_token_cost, :decimal, precision: 10, scale: 6 11 | add_column :raif_model_completions, :output_token_cost, :decimal, precision: 10, scale: 6 12 | add_column :raif_model_completions, :total_cost, :decimal, precision: 10, scale: 6 13 | end 14 | end 15 | -------------------------------------------------------------------------------- /lib/raif/evals/eval.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Evals 5 | class Eval 6 | attr_reader :description, :expectation_results 7 | 8 | def initialize(description:) 9 | @description = description 10 | @expectation_results = [] 11 | end 12 | 13 | def add_expectation_result(result) 14 | @expectation_results << result 15 | end 16 | 17 | def passed? 18 | expectation_results.all?(&:passed?) 19 | end 20 | 21 | def to_h 22 | { 23 | description: description, 24 | passed: passed?, 25 | expectation_results: expectation_results.map(&:to_h) 26 | } 27 | end 28 | end 29 | end 30 | end 31 | -------------------------------------------------------------------------------- /app/controllers/raif/admin/stats/model_tool_invocations_controller.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Admin 5 | module Stats 6 | class ModelToolInvocationsController < Raif::Admin::ApplicationController 7 | def index 8 | @selected_period = params[:period] || "day" 9 | @time_range = get_time_range(@selected_period) 10 | 11 | @model_tool_invocation_count = Raif::ModelToolInvocation.where(created_at: @time_range).count 12 | 13 | @model_tool_invocation_stats_by_type = Raif::ModelToolInvocation 14 | .where(created_at: @time_range) 15 | .group(:tool_type) 16 | .count 17 | end 18 | end 19 | end 20 | end 21 | end 22 | -------------------------------------------------------------------------------- /lib/raif/cli/evals_setup.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | require "optparse" 4 | require_relative "base" 5 | 6 | module Raif 7 | module CLI 8 | class EvalsSetup < Base 9 | def run 10 | OptionParser.new do |opts| 11 | opts.banner = "Usage: raif evals:setup [options]" 12 | opts.on("-h", "--help", "Show this help message") do 13 | puts opts 14 | exit 15 | end 16 | end.parse!(args) 17 | 18 | # Load Rails application to use generators 19 | load_rails_application 20 | 21 | # Invoke the Rails generator 22 | require "rails/generators" 23 | Rails::Generators.invoke("raif:evals:setup", args) 24 | end 25 | end 26 | end 27 | end 28 | -------------------------------------------------------------------------------- /.claude/settings.local.json: -------------------------------------------------------------------------------- 1 | { 2 | "permissions": { 3 | "allow": [ 4 | "Bash(bin/rails generate migration:*)", 5 | "Bash(bin/rails:*)", 6 | "Bash(bundle exec rspec:*)", 7 | "Bash(bundle exec rspec:*)", 8 | "Bash(cp:*)", 9 | "Bash(cp:*)", 10 | "Bash(find:*)", 11 | "Bash(ls:*)", 12 | "Bash(mkdir:*)", 13 | "Bash(rails:*)", 14 | "Bash(bin/lint:*)", 15 | "Bash(bundle exec rails console:*)", 16 | "Bash(RAILS_ENV=test bundle exec rails console)", 17 | "Bash(bundle install)", 18 | "Bash(bundle exec i18n-tasks:*)", 19 | "Bash(bundle exec rails runner:*)", 20 | "Bash(bundle exec rubocop:*)" 21 | ], 22 | "deny": [], 23 | "defaultMode": "acceptEdits" 24 | } 25 | } 26 | -------------------------------------------------------------------------------- /app/models/raif/concerns/llms/open_ai_completions/response_tool_calls.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif::Concerns::Llms::OpenAiCompletions::ResponseToolCalls 4 | extend ActiveSupport::Concern 5 | 6 | def extract_response_tool_calls(resp) 7 | tool_calls = resp.dig("choices", 0, "message", "tool_calls") 8 | return if tool_calls.blank? 9 | 10 | tool_calls.map do |tool_call| 11 | { 12 | "provider_tool_call_id" => tool_call["id"], 13 | "name" => tool_call["function"]["name"], 14 | "arguments" => begin 15 | JSON.parse(tool_call["function"]["arguments"]) 16 | rescue JSON::ParserError 17 | tool_call["function"]["arguments"] 18 | end 19 | } 20 | end 21 | end 22 | end 23 | -------------------------------------------------------------------------------- /spec/dummy/config/initializers/inflections.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | # Be sure to restart your server when you modify this file. 4 | 5 | # Add new inflection rules using the following format. Inflections 6 | # are locale specific, and you may define rules for as many different 7 | # locales as you wish. All of these examples are active by default: 8 | # ActiveSupport::Inflector.inflections(:en) do |inflect| 9 | # inflect.plural /^(ox)$/i, "\\1en" 10 | # inflect.singular /^(ox)en/i, "\\1" 11 | # inflect.irregular "person", "people" 12 | # inflect.uncountable %w( fish sheep ) 13 | # end 14 | 15 | # These inflection rules are supported but not enabled by default: 16 | # ActiveSupport::Inflector.inflections(:en) do |inflect| 17 | # inflect.acronym "RESTful" 18 | # end 19 | -------------------------------------------------------------------------------- /spec/dummy/app/controllers/agents_controller.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class AgentsController < ApplicationController 4 | 5 | def index 6 | end 7 | 8 | def create 9 | agent = Raif::Agents::NativeToolCallingAgent.new( 10 | task: params[:task], 11 | available_model_tools: [Raif::ModelTools::WikipediaSearch, Raif::ModelTools::FetchUrl], 12 | creator: current_user 13 | ) 14 | 15 | agent.run! do |conversation_history_entry| 16 | Turbo::StreamsChannel.broadcast_append_to( 17 | :agents, 18 | target: "agent-progress", 19 | partial: "agents/conversation_history_entry", 20 | locals: { agent: agent, conversation_history_entry: conversation_history_entry } 21 | ) 22 | end 23 | 24 | head :no_content 25 | end 26 | end 27 | -------------------------------------------------------------------------------- /spec/fixtures/llm_responses/anthropic/developer_managed_fetch_url.json: -------------------------------------------------------------------------------- 1 | { 2 | "id": "msg_abc123", 3 | "type": "message", 4 | "role": "assistant", 5 | "model": "claude-3-5-haiku-20241022", 6 | "content": [ 7 | { 8 | "type": "text", 9 | "text": "I'll fetch the content of the Wall Street Journal homepage for you." 10 | }, 11 | { 12 | "type": "tool_use", 13 | "id": "toolu_abc123", 14 | "name": "fetch_url", 15 | "input": { 16 | "url": "https://www.wsj.com" 17 | } 18 | } 19 | ], 20 | "stop_reason": "tool_use", 21 | "stop_sequence": null, 22 | "usage": { 23 | "input_tokens": 364, 24 | "cache_creation_input_tokens": 0, 25 | "cache_read_input_tokens": 0, 26 | "output_tokens": 75, 27 | "service_tier": "standard" 28 | } 29 | } -------------------------------------------------------------------------------- /spec/factories/shared/tasks.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | FactoryBot.define do 4 | factory :raif_task, class: "Raif::Task" do 5 | sequence(:prompt){|i| "prompt #{i} #{SecureRandom.hex(3)}" } 6 | llm_model_key { Raif.available_llm_keys.sample.to_s } 7 | 8 | trait :completed do 9 | sequence(:raw_response){|i| "response #{i} #{SecureRandom.hex(3)}" } 10 | created_at { 1.minute.ago } 11 | started_at { 1.minute.ago } 12 | completed_at { 30.seconds.ago } 13 | end 14 | 15 | trait :failed do 16 | created_at { 1.minute.ago } 17 | started_at { 1.minute.ago } 18 | failed_at { 30.seconds.ago } 19 | end 20 | end 21 | 22 | factory :raif_test_task, parent: :raif_task, class: "Raif::TestTask" do 23 | type { "Raif::TestTask" } 24 | end 25 | end 26 | -------------------------------------------------------------------------------- /spec/support/test_llm.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Llms 5 | class TestLlm < Raif::Llm 6 | include Raif::Concerns::Llms::OpenAiCompletions::MessageFormatting 7 | 8 | attr_accessor :chat_handler 9 | 10 | def perform_model_completion!(model_completion) 11 | result = chat_handler.call(model_completion.messages, model_completion) 12 | model_completion.raw_response = result if result.is_a?(String) 13 | model_completion.completion_tokens = rand(100..2000) 14 | model_completion.prompt_tokens = rand(100..2000) 15 | model_completion.total_tokens = model_completion.completion_tokens + model_completion.prompt_tokens 16 | model_completion.save! 17 | 18 | model_completion 19 | end 20 | end 21 | end 22 | end 23 | -------------------------------------------------------------------------------- /docs/_learn_more/demo_app.md: -------------------------------------------------------------------------------- 1 | --- 2 | layout: default 3 | title: Demo App 4 | nav_order: 9 5 | description: "Raif demo application" 6 | --- 7 | 8 | # Demo App 9 | 10 | Raif includes a [demo app](https://github.com/CultivateLabs/raif_demo) that you can use to see the engine in action. Assuming you have Ruby 3.4.2 and Postgres installed, you can run the demo app with: 11 | 12 | ```bash 13 | git clone git@github.com:CultivateLabs/raif_demo.git 14 | cd raif_demo 15 | bundle install 16 | bin/rails db:create db:prepare 17 | OPENAI_API_KEY=your-openai-api-key-here bin/rails s 18 | ``` 19 | 20 | You can then access the app at [http://localhost:3000](http://localhost:3000) 21 | 22 | ![Demo App Screenshot](../assets/images/screenshots/demo-app.png){:class="img-border"} 23 | 24 | --- 25 | 26 | **Read next:** [Tasks](../key_raif_concepts/tasks) -------------------------------------------------------------------------------- /spec/dummy/app/assets/stylesheets/application.css: -------------------------------------------------------------------------------- 1 | /* 2 | * This is a manifest file that'll be compiled into application.css, which will include all the files 3 | * listed below. 4 | * 5 | * Any CSS and SCSS file within this directory, lib/assets/stylesheets, vendor/assets/stylesheets, 6 | * or any plugin's vendor/assets/stylesheets directory can be referenced here using a relative path. 7 | * 8 | * You're free to add application-wide styles to this file and they'll appear at the bottom of the 9 | * compiled file so the styles you add here take precedence over styles defined in any other CSS/SCSS 10 | * files in this directory. Styles in this file should be added after the last require_* statement. 11 | * It is generally better to create a new file per style scope. 12 | * 13 | *= require_tree . 14 | *= require_self 15 | */ 16 | -------------------------------------------------------------------------------- /app/models/raif/concerns/llms/open_ai_responses/response_tool_calls.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif::Concerns::Llms::OpenAiResponses::ResponseToolCalls 4 | extend ActiveSupport::Concern 5 | 6 | def extract_response_tool_calls(resp) 7 | return if resp["output"].blank? 8 | 9 | tool_calls = [] 10 | resp["output"].each do |output_item| 11 | next unless output_item["type"] == "function_call" 12 | 13 | tool_calls << { 14 | "provider_tool_call_id" => output_item["call_id"], 15 | "name" => output_item["name"], 16 | "arguments" => begin 17 | JSON.parse(output_item["arguments"]) 18 | rescue JSON::ParserError 19 | output_item["arguments"] 20 | end 21 | } 22 | end 23 | 24 | tool_calls.any? ? tool_calls : nil 25 | end 26 | end 27 | -------------------------------------------------------------------------------- /lib/generators/raif/eval_set/eval_set_generator.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | require_relative "../base_generator" 4 | 5 | module Raif 6 | module Generators 7 | class EvalSetGenerator < BaseGenerator 8 | source_root File.expand_path("templates", __dir__) 9 | 10 | def create_eval_set_file 11 | template "eval_set.rb.tt", eval_set_file_path 12 | end 13 | 14 | def show_instructions 15 | say "\nEval set created!" 16 | say "To run this eval set: bundle exec raif evals ./#{eval_set_file_path}" 17 | say "To run all eval sets: bundle exec raif evals" 18 | say "" 19 | end 20 | 21 | private 22 | 23 | def eval_set_file_path 24 | File.join("raif_evals", "eval_sets", class_path, "#{file_name}_eval_set.rb") 25 | end 26 | end 27 | end 28 | end 29 | -------------------------------------------------------------------------------- /spec/support/test_embedding_model.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module EmbeddingModels 5 | class Test < Raif::EmbeddingModel 6 | attr_accessor :embedding_handler 7 | 8 | def generate_embedding!(input, dimensions: nil) 9 | if input.is_a?(Array) 10 | input.map { |text| generate_test_embedding!(text, dimensions:) } 11 | else 12 | generate_test_embedding!(input, dimensions:) 13 | end 14 | end 15 | 16 | def generate_test_embedding!(input, dimensions: nil) 17 | if embedding_handler.present? 18 | embedding_handler.call(input, dimensions) 19 | else 20 | dimensions ||= default_output_vector_size 21 | Array.new(dimensions) { rand(-1.0..1.0) } 22 | end 23 | end 24 | 25 | end 26 | end 27 | end 28 | -------------------------------------------------------------------------------- /app/views/raif/conversations/index.html.erb: -------------------------------------------------------------------------------- 1 |
2 |

<%= t("raif.conversations.index.title") %>

3 | 4 | <% if @conversations.any? %> 5 |
6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | <%= render collection: @conversations, partial: "raif/conversations/conversation" %> 15 | 16 |
<%= t("raif.conversations.index.table.started") %><%= t("raif.conversations.index.table.actions") %>
17 |
18 | <% else %> 19 |
20 |

<%= t("raif.conversations.index.no_conversations") %>

21 |
22 | <% end %> 23 |
24 | -------------------------------------------------------------------------------- /app/models/raif/concerns/llms/bedrock/response_tool_calls.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif::Concerns::Llms::Bedrock::ResponseToolCalls 4 | extend ActiveSupport::Concern 5 | 6 | def extract_response_tool_calls(resp) 7 | # Get the message from the response object 8 | message = resp.output.message 9 | return if message.content.nil? 10 | 11 | # Find any tool_use blocks in the content array 12 | tool_uses = message.content.select do |content| 13 | content.respond_to?(:tool_use) && content.tool_use.present? 14 | end 15 | 16 | return if tool_uses.blank? 17 | 18 | tool_uses.map do |content| 19 | { 20 | "provider_tool_call_id" => content.tool_use.tool_use_id, 21 | "name" => content.tool_use.name, 22 | "arguments" => content.tool_use.input 23 | } 24 | end 25 | end 26 | end 27 | -------------------------------------------------------------------------------- /spec/support/test_model_tool.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class Raif::TestModelTool < Raif::ModelTool 4 | tool_arguments_schema do 5 | array :items do 6 | object do 7 | string :title, description: "The title of the item" 8 | string :description 9 | end 10 | end 11 | end 12 | 13 | example_model_invocation do 14 | { 15 | "name": tool_name, 16 | "arguments": { "items": [{ "title": "foo", "description": "bar" }] } 17 | } 18 | end 19 | 20 | def self.process_invocation(tool_arguments) 21 | end 22 | 23 | tool_description do 24 | "Mock Tool Description" 25 | end 26 | 27 | def self.observation_for_invocation(tool_invocation) 28 | return if tool_invocation.result.blank? 29 | 30 | "Mock Observation for #{tool_invocation.id}. Result was: #{tool_invocation.result["status"]}" 31 | end 32 | end 33 | -------------------------------------------------------------------------------- /spec/dummy/config/initializers/raif.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | Raif.configure do |config| 4 | # config.conversations_controller = "ConversationsController" 5 | # config.conversation_entries_controller = "ConversationEntriesController" 6 | # config.conversation_llm_messages_max_length_default = 50 7 | 8 | config.anthropic_api_key = "placeholder" 9 | config.anthropic_models_enabled = true 10 | 11 | config.open_ai_api_key = "placeholder" 12 | config.open_ai_models_enabled = true 13 | config.open_ai_embedding_models_enabled = true 14 | 15 | config.open_router_api_key = "placeholder" 16 | config.open_router_models_enabled = true 17 | 18 | config.bedrock_embedding_models_enabled = true 19 | config.bedrock_models_enabled = true 20 | 21 | config.authorize_controller_action = ->() { true } 22 | config.authorize_admin_controller_action = ->() { true } 23 | end 24 | -------------------------------------------------------------------------------- /.erb_lint.yml: -------------------------------------------------------------------------------- 1 | --- 2 | EnableDefaultLinters: true 3 | exclude: 4 | - '**/vendor/**/*' 5 | - '**/node_modules/**/*' 6 | linters: 7 | RequireScriptNonce: 8 | enabled: true 9 | ErbSafety: 10 | enabled: true 11 | # better_html_config: .better-html.yml 12 | HardCodedString: 13 | enabled: true 14 | StrictLocals: 15 | enabled: false 16 | Rubocop: 17 | enabled: true 18 | rubocop_config: 19 | inherit_from: 20 | - .rubocop.yml 21 | Layout/InitialIndentation: 22 | Enabled: false 23 | Layout/LineLength: 24 | Enabled: false 25 | Layout/TrailingEmptyLines: 26 | Enabled: false 27 | Layout/TrailingWhitespace: 28 | Enabled: false 29 | Naming/FileName: 30 | Enabled: false 31 | Style/FrozenStringLiteralComment: 32 | Enabled: false 33 | Lint/UselessAssignment: 34 | Enabled: false -------------------------------------------------------------------------------- /lib/generators/raif/views_generator.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | class ViewsGenerator < Rails::Generators::Base 5 | source_root File.expand_path("../../../app/views/raif", __dir__) 6 | 7 | desc "Copies Raif conversation views to your application for customization" 8 | 9 | def copy_views 10 | directory "conversations", "app/views/raif/conversations" 11 | directory "conversation_entries", "app/views/raif/conversation_entries" 12 | end 13 | 14 | def success_message 15 | say_status :success, "Raif conversation views have been copied to your application", :green 16 | say "\nYou can now customize these views in:" 17 | say " app/views/raif/conversations/" 18 | say " app/views/raif/conversation_entries/" 19 | say "\nNote: These views will now override the default Raif engine views." 20 | end 21 | end 22 | end 23 | -------------------------------------------------------------------------------- /spec/dummy/config/routes.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | Rails.application.routes.draw do 4 | # Define your application routes per the DSL in https://guides.rubyonrails.org/routing.html 5 | 6 | # Reveal health status on /up that returns 200 if the app boots with no exceptions, otherwise 500. 7 | # Can be used by load balancers and uptime monitors to verify that the app is live. 8 | get "up" => "rails/health#show", as: :rails_health_check 9 | 10 | # Render dynamic PWA files from app/views/pwa/* (remember to link manifest in application.html.erb) 11 | # get "manifest" => "rails/pwa#manifest", as: :pwa_manifest 12 | # get "service-worker" => "rails/pwa#service_worker", as: :pwa_service_worker 13 | 14 | # Chat routes 15 | get "chat" => "chat#index", as: :chat 16 | 17 | resources :agents, only: [:create, :index] 18 | 19 | # Mount the Raif engine 20 | mount Raif::Engine, at: "/raif" 21 | end 22 | -------------------------------------------------------------------------------- /spec/models/raif/embedding_model_spec.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | require "rails_helper" 4 | 5 | RSpec.describe Raif::EmbeddingModel, type: :model do 6 | it "generates test embeddings" do 7 | model = Raif.embedding_model(:raif_test_embedding_model) 8 | embedding = model.generate_embedding!("Hello, world!") 9 | expect(embedding).to be_a(Array) 10 | expect(embedding.size).to eq(1536) 11 | end 12 | 13 | it "defaults to raif_test_embedding_model in test environment" do 14 | expect(Raif.default_embedding_model_key).to eq(:raif_test_embedding_model) 15 | end 16 | 17 | it "has model names for all built in embedding models" do 18 | Raif.default_embedding_models.values.flatten.each do |embedding_model_config| 19 | embedding_model = Raif.embedding_model(embedding_model_config[:key]) 20 | expect(embedding_model.name).to_not include("Translation missing") 21 | end 22 | end 23 | end 24 | -------------------------------------------------------------------------------- /app/controllers/raif/admin/model_tool_invocations_controller.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Admin 5 | class ModelToolInvocationsController < Raif::Admin::ApplicationController 6 | include Pagy::Backend 7 | 8 | def index 9 | @tool_types = Raif::ModelToolInvocation.distinct.pluck(:tool_type) 10 | @selected_type = params[:tool_types].present? && @tool_types.include?(params[:tool_types]) ? params[:tool_types] : "all" 11 | 12 | model_tool_invocations = Raif::ModelToolInvocation.newest_first 13 | model_tool_invocations = model_tool_invocations.where(tool_type: @selected_type) if @selected_type.present? && @selected_type != "all" 14 | 15 | @pagy, @model_tool_invocations = pagy(model_tool_invocations) 16 | end 17 | 18 | def show 19 | @model_tool_invocation = Raif::ModelToolInvocation.find(params[:id]) 20 | end 21 | end 22 | end 23 | end 24 | -------------------------------------------------------------------------------- /lib/raif/evals/llm_judge.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Evals 5 | class LlmJudge < Raif::Task 6 | # Set default temperature for consistent judging 7 | llm_temperature 0.0 8 | 9 | # Default to JSON response format for structured output 10 | llm_response_format :json 11 | 12 | run_with :content_to_judge # the content to judge 13 | run_with :additional_context # additional context to be provided to the judge 14 | 15 | def default_llm_model_key 16 | Raif.config.evals_default_llm_judge_model_key || super 17 | end 18 | 19 | def judgment_reasoning 20 | parsed_response["reasoning"] if completed? 21 | end 22 | 23 | def judgment_confidence 24 | parsed_response["confidence"] if completed? 25 | end 26 | 27 | def low_confidence? 28 | judgment_confidence && judgment_confidence < 0.5 29 | end 30 | end 31 | end 32 | end 33 | -------------------------------------------------------------------------------- /spec/factories/shared/model_completions.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | FactoryBot.define do 4 | factory :raif_model_completion, class: "Raif::ModelCompletion" do 5 | llm_model_key { Raif.available_llm_keys.sample.to_s } 6 | response_format { Raif::Llm.valid_response_formats.sample.to_s } 7 | sequence(:raw_response) { |i| "Model response #{i} #{SecureRandom.hex(4)}" } 8 | messages { [{ "role" => "user", "content" => "Test message" }] } 9 | prompt_tokens { rand(10..50) } 10 | completion_tokens { rand(20..100) } 11 | total_tokens { prompt_tokens + completion_tokens } 12 | 13 | trait :with_json_response do 14 | response_format { "json" } 15 | raw_response { '{"message": "This is a JSON response", "data": {"key": "value"}}' } 16 | end 17 | 18 | trait :with_html_response do 19 | response_format { "html" } 20 | raw_response { '
This is an HTML response
' } 21 | end 22 | end 23 | end 24 | -------------------------------------------------------------------------------- /lib/generators/raif/model_tool/model_tool_generator.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | require_relative "../base_generator" 4 | 5 | module Raif 6 | module Generators 7 | class ModelToolGenerator < BaseGenerator 8 | source_root File.expand_path("templates", __dir__) 9 | 10 | desc "Creates a new model tool for the LLM to invoke in app/models/raif/model_tools" 11 | 12 | def create_model_tool_file 13 | template "model_tool.rb.tt", File.join("app/models/raif/model_tools", class_path, "#{file_name}.rb") 14 | template "model_tool_invocation_partial.html.erb.tt", File.join("app/views/raif/model_tool_invocations", class_path, "_#{file_name}.html.erb") 15 | end 16 | 17 | def success_message 18 | say_status :success, "Model tool created successfully", :green 19 | say "\nYou can now implement your model tool in:" 20 | say " app/models/raif/model_tools/#{file_name}.rb" 21 | end 22 | 23 | end 24 | end 25 | end 26 | -------------------------------------------------------------------------------- /app/models/raif/model_tools/agent_final_answer.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class Raif::ModelTools::AgentFinalAnswer < Raif::ModelTool 4 | tool_arguments_schema do 5 | string "final_answer", description: "Your complete and final answer to the user's question or task" 6 | end 7 | 8 | example_model_invocation do 9 | { 10 | "name" => tool_name, 11 | "arguments" => { "final_answer": "The answer to the user's question or task" } 12 | } 13 | end 14 | 15 | tool_description do 16 | "Provide your final answer to the user's question or task" 17 | end 18 | 19 | class << self 20 | def observation_for_invocation(tool_invocation) 21 | return "No answer provided" unless tool_invocation.result.present? 22 | 23 | tool_invocation.result 24 | end 25 | 26 | def process_invocation(tool_invocation) 27 | tool_invocation.update!(result: tool_invocation.tool_arguments["final_answer"]) 28 | tool_invocation.result 29 | end 30 | end 31 | 32 | end 33 | -------------------------------------------------------------------------------- /lib/raif/evals/expectation_result.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Evals 5 | class ExpectationResult 6 | attr_reader :description, :status, :error 7 | attr_accessor :metadata, :error_message 8 | 9 | def initialize(description:, status:, error: nil, error_message: nil, metadata: nil) 10 | @description = description 11 | @status = status 12 | @error = error 13 | @error_message = error_message 14 | @metadata = metadata 15 | end 16 | 17 | def passed? 18 | @status == :passed 19 | end 20 | 21 | def failed? 22 | @status == :failed 23 | end 24 | 25 | def error? 26 | @status == :error 27 | end 28 | 29 | def to_h 30 | { 31 | description: description, 32 | status: status, 33 | error: error_message.presence || error&.message, 34 | metadata: metadata 35 | }.compact 36 | end 37 | end 38 | end 39 | end 40 | -------------------------------------------------------------------------------- /spec/dummy/app/views/pwa/service-worker.js: -------------------------------------------------------------------------------- 1 | // Add a service worker for processing Web Push notifications: 2 | // 3 | // self.addEventListener("push", async (event) => { 4 | // const { title, options } = await event.data.json() 5 | // event.waitUntil(self.registration.showNotification(title, options)) 6 | // }) 7 | // 8 | // self.addEventListener("notificationclick", function(event) { 9 | // event.notification.close() 10 | // event.waitUntil( 11 | // clients.matchAll({ type: "window" }).then((clientList) => { 12 | // for (let i = 0; i < clientList.length; i++) { 13 | // let client = clientList[i] 14 | // let clientPath = (new URL(client.url)).pathname 15 | // 16 | // if (clientPath == event.notification.data.path && "focus" in client) { 17 | // return client.focus() 18 | // } 19 | // } 20 | // 21 | // if (clients.openWindow) { 22 | // return clients.openWindow(event.notification.data.path) 23 | // } 24 | // }) 25 | // ) 26 | // }) 27 | -------------------------------------------------------------------------------- /lib/raif/cli/base.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module CLI 5 | class Base 6 | attr_reader :args, :options 7 | 8 | def initialize(args = []) 9 | @args = args 10 | @options = {} 11 | end 12 | 13 | protected 14 | 15 | def find_rails_root 16 | current = Dir.pwd 17 | 18 | until File.exist?(File.join(current, "config", "environment.rb")) 19 | parent = File.dirname(current) 20 | if parent == current 21 | puts "Error: Could not find Rails application root" 22 | puts "Please run this command from within a Rails application directory" 23 | exit 1 24 | end 25 | 26 | current = parent 27 | end 28 | 29 | current 30 | end 31 | 32 | def load_rails_application 33 | rails_root = find_rails_root 34 | Dir.chdir(rails_root) 35 | require File.join(rails_root, "config", "environment") 36 | end 37 | end 38 | end 39 | end 40 | -------------------------------------------------------------------------------- /lib/raif.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | require "raif/version" 4 | require "raif/languages" 5 | require "raif/engine" 6 | require "raif/configuration" 7 | require "raif/errors" 8 | require "raif/utils" 9 | require "raif/llm_registry" 10 | require "raif/embedding_model_registry" 11 | require "raif/json_schema_builder" 12 | require "raif/migration_checker" 13 | require "raif/messages" 14 | 15 | require "faraday" 16 | require "event_stream_parser" 17 | require "json-schema" 18 | require "loofah" 19 | require "pagy" 20 | require "reverse_markdown" 21 | require "turbo-rails" 22 | 23 | module Raif 24 | class << self 25 | attr_accessor :configuration 26 | 27 | attr_writer :logger 28 | end 29 | 30 | def self.config 31 | @configuration ||= Raif::Configuration.new 32 | end 33 | 34 | def self.configure 35 | yield(config) 36 | end 37 | 38 | def self.logger 39 | @logger ||= Rails.logger 40 | end 41 | 42 | def self.running_evals? 43 | ENV["RAIF_RUNNING_EVALS"] == "true" 44 | end 45 | end 46 | -------------------------------------------------------------------------------- /spec/support/current_temperature_test_tool.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class Raif::ModelTools::CurrentTemperatureTestTool < Raif::ModelTool 4 | tool_arguments_schema do 5 | string :zip_code, description: "The zip code to get the current temperature for" 6 | end 7 | 8 | tool_description do 9 | "A tool to get the current temperature for a given zip code" 10 | end 11 | 12 | class << self 13 | def process_invocation(tool_invocation) 14 | tool_invocation.update!( 15 | result: { 16 | temperature: 72 17 | } 18 | ) 19 | 20 | tool_invocation.result 21 | end 22 | 23 | def triggers_observation_to_model? 24 | true 25 | end 26 | 27 | def observation_for_invocation(tool_invocation) 28 | zip_code = tool_invocation.tool_arguments["zip_code"] 29 | temperature = tool_invocation.result["temperature"] 30 | 31 | "The current temperature for zip code #{zip_code} is #{temperature} degrees Fahrenheit." 32 | end 33 | end 34 | end 35 | -------------------------------------------------------------------------------- /app/views/raif/admin/tasks/_task.html.erb: -------------------------------------------------------------------------------- 1 | 2 | <%= link_to "##{task.id}", raif.admin_task_path(task) %> 3 | <%= task.type %> 4 | <%= task.created_at.rfc822 %> 5 | <%= task.creator.try(:raif_display_name) || "#{task.creator_type} ##{task.creator_id}" %> 6 | <%= task.llm_model_key %> 7 | 8 | <% case task.status %> 9 | <% when :completed %> 10 | <%= t("raif.admin.common.completed") %> 11 | <% when :failed %> 12 | <%= t("raif.admin.common.failed") %> 13 | <% when :in_progress %> 14 | <%= t("raif.admin.common.in_progress") %> 15 | <% else %> 16 | <%= t("raif.admin.common.pending") %> 17 | <% end %> 18 | 19 | <%= truncate(task.prompt, length: 100) %> 20 | 21 | -------------------------------------------------------------------------------- /app/views/raif/conversation_entries/_form_with_user_tool_invocation.html.erb: -------------------------------------------------------------------------------- 1 |
2 |
3 |
4 |
<%= conversation_entry.raif_user_tool_invocation.tool_name %>
5 | 6 | <%= link_to raif.new_conversation_entry_path(conversation), class: "text-dark", data: { turbo_stream: true } do %> 7 | 8 | 9 | 10 | <% end %> 11 |
12 |
13 |
14 | <%= render "raif/conversation_entries/form", 15 | conversation: conversation, 16 | conversation_entry: conversation_entry %> 17 |
18 |
19 | -------------------------------------------------------------------------------- /app/views/raif/admin/model_tool_invocations/_model_tool_invocation.html.erb: -------------------------------------------------------------------------------- 1 | 2 | <%= link_to "##{model_tool_invocation.id}", raif.admin_model_tool_invocation_path(model_tool_invocation) %> 3 | <%= model_tool_invocation.created_at.rfc822 %> 4 | <%= model_tool_invocation.source_type %> #<%= model_tool_invocation.source_id %> 5 | <%= model_tool_invocation.tool_type.demodulize %> 6 | 7 | <% if model_tool_invocation.completed_at? %> 8 | <%= t("raif.admin.common.completed") %> 9 | <% elsif model_tool_invocation.failed_at? %> 10 | <%= t("raif.admin.common.failed") %> 11 | <% else %> 12 | <%= t("raif.admin.common.pending") %> 13 | <% end %> 14 | 15 | <%= truncate(model_tool_invocation.tool_arguments.to_json, length: 100) %> 16 | 17 | -------------------------------------------------------------------------------- /config/routes.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | Raif::Engine.routes.draw do 4 | resources :conversations, 5 | only: [:index, :show], 6 | controller: "/#{Raif.config.conversations_controller.constantize.controller_path}" do 7 | resources :conversation_entries, 8 | only: [:new, :create], 9 | as: :entries, 10 | path: "entries", 11 | controller: "/#{Raif.config.conversation_entries_controller.constantize.controller_path}" 12 | end 13 | 14 | namespace :admin do 15 | root to: redirect("admin/model_completions") 16 | resources :stats, only: [:index] 17 | 18 | namespace :stats do 19 | resources :tasks, only: [:index] 20 | resources :model_tool_invocations, only: [:index] 21 | end 22 | 23 | resources :tasks, only: [:index, :show] 24 | resources :conversations, only: [:index, :show] 25 | resources :model_completions, only: [:index, :show] 26 | resources :agents, only: [:index, :show] 27 | resources :model_tool_invocations, only: [:index, :show] 28 | resource :config, only: [:show] 29 | end 30 | end 31 | -------------------------------------------------------------------------------- /spec/dummy/app/views/layouts/application.html.erb: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | <%= content_for(:title) || "Dummy" %> 5 | 6 | 7 | 8 | <%= csrf_meta_tags %> 9 | <%= csp_meta_tag %> 10 | 11 | <%= yield :head %> 12 | 13 | <%# Enable PWA manifest for installable apps (make sure to enable in config/routes.rb too!) %> 14 | <%#= tag.link rel: "manifest", href: pwa_manifest_path(format: :json) %> 15 | 16 | 17 | 18 | 19 | 20 | <%# Includes all stylesheet files in app/assets/stylesheets %> 21 | <%= stylesheet_link_tag "application" %> 22 | <%= stylesheet_link_tag "raif" %> 23 | <%= javascript_importmap_tags %> 24 | 25 | 26 | 27 | <%= yield %> 28 | 29 | 30 | -------------------------------------------------------------------------------- /app/views/raif/admin/model_completions/_model_completion.html.erb: -------------------------------------------------------------------------------- 1 | 2 | <%= link_to "##{model_completion.id}", raif.admin_model_completion_path(model_completion) %> 3 | <%= model_completion.created_at.rfc822 %> 4 | <%= model_completion.source_type %> #<%= model_completion.source_id %> 5 | <%= model_completion.llm_model_key %> 6 | <%= model_completion.response_format %> 7 | <%= model_completion.total_tokens ? number_with_delimiter(model_completion.total_tokens) : "-" %> 8 | <%= model_completion.total_cost ? number_to_currency(model_completion.total_cost, precision: 6) : "-" %> 9 | 10 | <% if model_completion.citations.present? %> 11 | <%= model_completion.citations.length %> 12 | <% else %> 13 | - 14 | <% end %> 15 | 16 | <%= truncate(model_completion.raw_response, length: 100) %> 17 | 18 | -------------------------------------------------------------------------------- /bin/lint: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env sh 2 | 3 | # Check if the -a flag is passed 4 | if [ "$1" = "-a" ]; then 5 | RUBOCOP_CMD="rubocop -a" 6 | ERBLINT_CMD="bundle exec erb_lint --lint-all -a" 7 | I18N_CMD="bundle exec i18n-tasks normalize" 8 | else 9 | RUBOCOP_CMD="rubocop" 10 | ERBLINT_CMD="bundle exec erb_lint --cache --lint-all" 11 | I18N_CMD="bundle exec i18n-tasks health" 12 | fi 13 | 14 | # Set initial exit code 15 | EXIT_CODE=0 16 | 17 | echo "==> Running rubocop\n" 18 | $RUBOCOP_CMD 19 | if [ $? -ne 0 ]; then 20 | EXIT_CODE=1 21 | fi 22 | 23 | echo "\n\n==> Running erb lint\n" 24 | $ERBLINT_CMD 25 | if [ $? -ne 0 ]; then 26 | EXIT_CODE=1 27 | fi 28 | 29 | echo "\n\n==> Running i18n-tasks\n" 30 | $I18N_CMD 31 | if [ $? -ne 0 ]; then 32 | EXIT_CODE=1 33 | fi 34 | 35 | if [ "$1" = "-a" ]; then 36 | echo "\n\n==> Running i18n-tasks health\n" 37 | bundle exec i18n-tasks health 38 | if [ $? -ne 0 ]; then 39 | EXIT_CODE=1 40 | fi 41 | fi 42 | 43 | if [ $EXIT_CODE -eq 0 ]; then 44 | echo "\n\n" 45 | echo "\033[32m✅ All linting passed! ✅\033[0m" 46 | echo "\n\n" 47 | fi 48 | 49 | exit $EXIT_CODE -------------------------------------------------------------------------------- /spec/dummy/config/locales/en.yml: -------------------------------------------------------------------------------- 1 | # Files in the config/locales directory are used for internationalization and 2 | # are automatically loaded by Rails. If you want to use locales other than 3 | # English, add the necessary files in this directory. 4 | # 5 | # To use the locales, use `I18n.t`: 6 | # 7 | # I18n.t "hello" 8 | # 9 | # In views, this is aliased to just `t`: 10 | # 11 | # <%= t("hello") %> 12 | # 13 | # To use a different locale, set it with `I18n.locale`: 14 | # 15 | # I18n.locale = :es 16 | # 17 | # This would use the information in config/locales/es.yml. 18 | # 19 | # To learn more about the API, please read the Rails Internationalization guide 20 | # at https://guides.rubyonrails.org/i18n.html. 21 | # 22 | # Be aware that YAML interprets the following case-insensitive strings as 23 | # booleans: `true`, `false`, `on`, `off`, `yes`, `no`. Therefore, these strings 24 | # must be quoted to be interpreted as strings. For example: 25 | # 26 | # en: 27 | # "yes": yup 28 | # enabled: "ON" 29 | 30 | en: 31 | dummy: 32 | tool_call: "tool_call" 33 | tool_result: "tool_result" 34 | -------------------------------------------------------------------------------- /MIT-LICENSE: -------------------------------------------------------------------------------- 1 | Copyright Ben Roesch 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining 4 | a copy of this software and associated documentation files (the 5 | "Software"), to deal in the Software without restriction, including 6 | without limitation the rights to use, copy, modify, merge, publish, 7 | distribute, sublicense, and/or sell copies of the Software, and to 8 | permit persons to whom the Software is furnished to do so, subject to 9 | the following conditions: 10 | 11 | The above copyright notice and this permission notice shall be 12 | included in all copies or substantial portions of the Software. 13 | 14 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 15 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 16 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 17 | NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE 18 | LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 19 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION 20 | WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 21 | -------------------------------------------------------------------------------- /app/models/raif/concerns/llms/open_ai_completions/tool_formatting.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif::Concerns::Llms::OpenAiCompletions::ToolFormatting 4 | extend ActiveSupport::Concern 5 | 6 | def build_tools_parameter(model_completion) 7 | model_completion.available_model_tools_map.map do |_tool_name, tool| 8 | if tool.provider_managed? 9 | raise Raif::Errors::UnsupportedFeatureError, 10 | "Raif doesn't yet support provider-managed tools for the OpenAI Completions API. Consider using the OpenAI Responses API instead." 11 | else 12 | # It's a developer-managed tool 13 | validate_json_schema!(tool.tool_arguments_schema) 14 | 15 | { 16 | type: "function", 17 | function: { 18 | name: tool.tool_name, 19 | description: tool.tool_description, 20 | parameters: tool.tool_arguments_schema 21 | } 22 | } 23 | end 24 | end 25 | end 26 | 27 | def build_forced_tool_choice(tool_name) 28 | { "type" => "function", "function" => { "name" => tool_name } } 29 | end 30 | end 31 | -------------------------------------------------------------------------------- /app/models/raif/embedding_models/bedrock.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class Raif::EmbeddingModels::Bedrock < Raif::EmbeddingModel 4 | 5 | def generate_embedding!(input, dimensions: nil) 6 | unless input.is_a?(String) 7 | raise ArgumentError, "Raif::EmbeddingModels::Bedrock#generate_embedding! input must be a string" 8 | end 9 | 10 | params = build_request_parameters(input, dimensions:) 11 | response = bedrock_client.invoke_model(params) 12 | 13 | response_body = JSON.parse(response.body.read) 14 | response_body["embedding"] 15 | rescue Aws::BedrockRuntime::Errors::ServiceError => e 16 | raise "Bedrock API error: #{e.message}" 17 | end 18 | 19 | private 20 | 21 | def build_request_parameters(input, dimensions: nil) 22 | body_params = { inputText: input } 23 | body_params[:dimensions] = dimensions if dimensions.present? 24 | 25 | { 26 | model_id: api_name, 27 | body: body_params.to_json 28 | } 29 | end 30 | 31 | def bedrock_client 32 | @bedrock_client ||= Aws::BedrockRuntime::Client.new(region: Raif.config.aws_bedrock_region) 33 | end 34 | end 35 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing to Raif 2 | 3 | ## Development Environment Setup 4 | 5 | ### Setup Steps 6 | 7 | 1. **Fork and clone the repository:** 8 | Fork the repository first, then clone it to your local machine: 9 | 10 | ```bash 11 | git clone https://github.com/YOUR_USERNAME/raif.git 12 | cd raif 13 | git remote add upstream https://github.com/CultivateLabs/raif.git 14 | git fetch upstream 15 | git checkout main 16 | ``` 17 | 18 | 2. **Install Ruby dependencies:** 19 | ```bash 20 | bundle install 21 | ``` 22 | 23 | 3. **Install JavaScript dependencies:** 24 | ```bash 25 | yarn install 26 | ``` 27 | 28 | 4. **Set up the database:** 29 | ```bash 30 | bin/rails db:setup 31 | ``` 32 | 33 | ## Running Tests 34 | 35 | Raif uses RSpec for testing. 36 | 37 | ### Run All Tests 38 | ```bash 39 | bundle exec rspec 40 | ``` 41 | 42 | ### Run Tests with Guard (auto-reload) 43 | ```bash 44 | bundle exec guard 45 | ``` 46 | 47 | ### Linting 48 | 49 | Raif uses Rubocop, ERB Lint, and i18n-tasks for linting. 50 | 51 | To run all linters: 52 | ```bash 53 | bin/lint 54 | ``` 55 | 56 | Or to lint with auto-correct: 57 | ```bash 58 | bin/lint -a 59 | ``` -------------------------------------------------------------------------------- /spec/models/raif/model_tool_spec.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | require "rails_helper" 4 | 5 | RSpec.describe Raif::ModelTool, type: :model do 6 | describe "tool_arguments_schema" do 7 | it "returns the tool_arguments_schema" do 8 | expect(Raif::TestModelTool.tool_arguments_schema).to eq({ 9 | type: "object", 10 | additionalProperties: false, 11 | required: ["items"], 12 | properties: { 13 | items: { 14 | type: "array", 15 | items: { 16 | type: "object", 17 | additionalProperties: false, 18 | properties: { 19 | title: { type: "string", description: "The title of the item" }, 20 | description: { type: "string" }, 21 | }, 22 | required: ["title", "description"], 23 | } 24 | } 25 | } 26 | }) 27 | end 28 | 29 | it "validates against OpenAI's rules" do 30 | llm = Raif.llm(:open_ai_gpt_4o_mini) 31 | expect(llm.validate_json_schema!(Raif::TestModelTool.tool_arguments_schema)).to eq(true) 32 | end 33 | end 34 | end 35 | -------------------------------------------------------------------------------- /app/models/raif/application_record.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | class Raif::ApplicationRecord < Raif.config.model_superclass.constantize 4 | include Raif::Concerns::BooleanTimestamp 5 | 6 | self.abstract_class = true 7 | 8 | scope :newest_first, -> { order(created_at: :desc) } 9 | scope :oldest_first, -> { order(created_at: :asc) } 10 | 11 | # Returns a scope that checks if a JSON column is not blank (not null and not empty array) 12 | # @param column_name [Symbol, String] the name of the JSON column 13 | # @return [ActiveRecord::Relation] 14 | def self.where_json_not_blank(column_name) 15 | quoted_column = connection.quote_column_name(column_name.to_s) 16 | 17 | case connection.adapter_name.downcase 18 | when "postgresql" 19 | where.not(column_name => nil) 20 | .where("jsonb_array_length(#{quoted_column}) > 0") 21 | when "mysql2", "trilogy" 22 | where.not(column_name => nil) 23 | .where("JSON_LENGTH(#{quoted_column}) > 0") 24 | else 25 | raise "Unsupported database: #{connection.adapter_name}" 26 | end 27 | end 28 | 29 | def self.table_name_prefix 30 | "raif_" 31 | end 32 | end 33 | -------------------------------------------------------------------------------- /spec/dummy/app/models/raif/conversations/html_conversation_with_tools.rb: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | module Raif 4 | module Conversations 5 | class HtmlConversationWithTools < Raif::ApplicationConversation 6 | llm_response_format :html 7 | 8 | before_create -> { self.available_model_tools = ["Raif::ModelTools::ProviderManaged::WebSearch"] } 9 | 10 | def system_prompt_intro 11 | <<~PROMPT.strip 12 | You are an expert songwriter. You are given a topic and you need to write a song about it. 13 | 14 | Your response should be formatted using basic HTML tags such as

,