├── .env.example ├── .gitignore ├── README.md ├── app.py ├── images ├── api_key.png ├── file_selection.png ├── question_answer.png └── results.png ├── requirements.txt └── utils └── __init__.py /.env.example: -------------------------------------------------------------------------------- 1 | ASSEMBLYAI_API_KEY=paste-your-key-here -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | venv 2 | .env 3 | */__pycache__ -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # LeMUR Lecture Summarizer 2 | 3 | This application allows you to automatically summarize lectures and ask questions about the lesson material. Learn how to build this application in the [associated blog](https://www.assemblyai.com/blog/build-an-interactive-lecture-summarization-app/). 4 | 5 | The application was tested with **Python 3.10.5** 6 | 7 | ## Prerequisites 8 | You must have: 9 | 1. [Python](https://www.python.org/) installed 10 | 2. [pip](https://pip.pypa.io/en/stable/installation/) installed 11 | 3. An [AssemblyAI](https://www.assemblyai.com/dashboard/signup) account 12 | 13 | ## Setup 14 | 15 | 1. Clone this repository and cd into it 16 | ```bash 17 | git clone https://github.com/AssemblyAI-Examples/lemur-lecture-summarizer.git 18 | cd lemur-lecture-summarizer 19 | ``` 20 | 21 | 2. Create and activate a virtual environment (optional) 22 | 23 | MacOS/Linux: 24 | ```bash 25 | python -m venv venv # you may need to use `python3` instead 26 | source ./venv/bin/activate 27 | ``` 28 | 29 | Windows: 30 | ```bash 31 | python -m venv venv # you may need to use `python3` instead 32 | .\venv\Scripts\activate.bat 33 | ``` 34 | 35 | 3. Install dependencies 36 | ```bash 37 | pip install -r requirements.txt 38 | ``` 39 | 40 | 4. Set your AssemblyAI API Key (optional) 41 | 42 | In the `.env` file, replace `paste-your-key-here` with your AssemblyAI API key, which you can copy from your [Dashboard](https://www.assemblyai.com/dashboard/login). If you do not do this, you will be required to enter your API key in the application. 43 | 44 | Note that you will need to have set up billing to use this application since it utilizes [LeMUR](https://www.assemblyai.com/blog/lemur/). 45 | 46 | ## Run the application 47 | 48 | 1. Start the app 49 | ```bash 50 | streamlit run app.py 51 | ``` 52 | 53 | 2. Open the app 54 | Click the link output in the terminal by the last command - the default is http://localhost:8501/ 55 | 56 | ## Use the application 57 | 58 | 1. Enter your AssemblyAI API key if you did not follow step 4 in the [Setup](#setup) section 59 | 60 | ![Entering your API key](images/api_key.png) 61 | 62 | 2. Select the lecture file 63 | 64 | You can use either an audio or video file, and the file can be locally stored, remotely stored (and publicly accessibly), or on YouTube. 65 | 66 | You can optionally add `Context` to provide contextualizing information about the lecture. 67 | 68 | ![Selecting a lecture file](images/file_selection.png) 69 | 70 | 3. View the results 71 | 72 | Click "Submit" and wait for the results. 73 | 74 | Processing time will depend on the length of the file - hour long lectures may take several minutes to process. 75 | 76 | ![Viewing the results](images/results.png) 77 | 78 | 4. Ask a question (optional) 79 | 80 | You can ask questions about the course content for further clarification 81 | 82 | ![Asking a question](images/question_answer.png) 83 | -------------------------------------------------------------------------------- /app.py: -------------------------------------------------------------------------------- 1 | import os 2 | 3 | import streamlit as st 4 | import assemblyai as aai 5 | from dotenv import load_dotenv 6 | load_dotenv() 7 | 8 | from utils import get_transcript, ask_question, return_ytdlp_fname 9 | 10 | environ_key = os.environ.get("ASSEMBLYAI_API_KEY") 11 | if environ_key is None: 12 | pass 13 | elif environ_key == "paste-your-key-here": 14 | environ_key = None 15 | else: 16 | aai.settings.api_key = environ_key 17 | 18 | # Remove existing temp files in case if improper shutdown 19 | temp_files = [f for f in os.listdir() if f.startswith('tmp')] 20 | for f in temp_files: 21 | os.remove(f) 22 | 23 | # constant 24 | YTDLP_FNAME = return_ytdlp_fname() 25 | 26 | # Setting defaults for conditional rendering 27 | input_key = None 28 | f = None 29 | entered = None 30 | summary = None 31 | question_submit = None 32 | answer = '' 33 | 34 | # Initializing state variables 35 | state_strings = ['summary', 'entered', 'transcript'] 36 | for s in state_strings: 37 | if s not in st.session_state: 38 | st.session_state[s] = None 39 | 40 | def set_aai_key(): 41 | """ Callback to change set AAI API key when the key is input in the text area """ 42 | aai.settings.api_key = st.session_state.input_aai_key 43 | 44 | # MAIN APPLICATION CONTENT 45 | 46 | "# Lecture Summarizer" 47 | "Use this application to **automatically summarize a virtual lecture** and **ask questions** about the lesson material." 48 | "Learn how to build this app [here](https://www.assemblyai.com/blog/build-an-interactive-lecture-summarization-app/)." 49 | 50 | with st.expander("Processing time"): 51 | "You can expect AssemblyAI's API to process a one hour lecture in less than a minute" 52 | "If a YouTube link is used, additional time will be required to extract the file." 53 | 54 | st.divider() 55 | 56 | if not environ_key: 57 | "## API Key" 58 | """ 59 | To get started, paste your AssemblyAI API key in the below box. 60 | If you don't have an API key, you can get one [here](https://www.assemblyai.com/dashboard/signup). You will need to set up billing in order to use this application since it uses [LeMUR](https://www.assemblyai.com/blog/lemur/). 61 | 62 | You can copy your API key by pressing the `Copy token` button on the right hand side of your [Dashboard](https://www.assemblyai.com/app). 63 | """ 64 | input_key = st.text_input( 65 | "API Key", 66 | placeholder="Enter your AssemblyAI API key here", 67 | type="password", 68 | on_change=set_aai_key, 69 | key='input_aai_key' 70 | ) 71 | 72 | st.warning("Note: You can avoid this section by setting the `ASSEMBLYAI_API_KEY` environment variable, either through the terminal or the `.env` file.", icon="🗒️") 73 | 74 | if input_key or environ_key: 75 | "## Lecture" 76 | """ 77 | Enter the lecture you would like to summarize below. You can use a local file on your computer, a remote file that is publicly-available online, or a YouTube video. 78 | """ 79 | 80 | # File type options 81 | ftype = st.radio("File type", ('Local file', 'Remote file', 'YouTube link')) 82 | 83 | if ftype == 'Local file': 84 | # Store the uploaded file in a temporary file 85 | f = st.file_uploader("File") 86 | if f: 87 | uploaded_ftype = f.name.split('.')[-1] 88 | temp_fname = f"tmp.{uploaded_ftype}" 89 | with open(temp_fname, 'wb') as fl: 90 | fl.write(f.read()) 91 | f = temp_fname 92 | elif ftype == 'Remote file': 93 | f = st.text_input("Link", 94 | value="https://storage.googleapis.com/aai-web-samples/cs50p-unit-tests.mp3", 95 | placeholder="Public link to the file" 96 | ) 97 | elif ftype == 'YouTube link': 98 | f = st.text_input("Link", 99 | value="https://www.youtube.com/watch?v=tIrcxwLqzjQ", 100 | placeholder="YouTube link" 101 | ) 102 | 103 | value = "" if ftype == "Local file" else "A lesson from Harvard's CS50P course. The lesson is about Unit Testing in Python." 104 | placeholder = "Contextualizing information about the file (optional)" 105 | context = st.text_input("Context", value=value, placeholder=placeholder) 106 | 107 | if f: 108 | entered = st.button("Submit") 109 | if entered: 110 | 111 | transcript = get_transcript(f, ftype) 112 | if ftype == "Local file": 113 | os.remove(f) 114 | elif ftype == "YouTube link": 115 | os.remove(YTDLP_FNAME) # remove file bc youtube DL will not work if there already exists file with that name 116 | 117 | st.session_state['transcript'] = transcript 118 | 119 | 120 | params = { 121 | 'answer_format': "****\n", 122 | 'max_output_size': 4000 123 | } 124 | if context: params['context'] = context 125 | 126 | with st.spinner("Generating summary..."): 127 | try: 128 | summary = transcript.lemur.summarize(**params) 129 | st.session_state['summary'] = summary.response.strip().split('\n') 130 | st.session_state['entered'] = True 131 | print('session summary: ', st.session_state['summary']) 132 | except aai.types.LemurError as e: 133 | st.write(f'Error: {str(e)}') 134 | st.session_state['entered'] = False 135 | 136 | if st.session_state['entered']: 137 | "## Results" 138 | 139 | if st.session_state['summary']: 140 | for i in st.session_state['summary']: 141 | st.markdown(i) 142 | 143 | 144 | if st.session_state['summary']: 145 | "# Questions" 146 | "Ask a question about the lesson below:" 147 | 148 | question = st.text_input("Question", 149 | placeholder="What is the point of using Pytest?", 150 | ) 151 | 152 | question_asked = st.button("Submit", key='question_asked') 153 | if question_asked: 154 | with st.spinner('Asking question...'): 155 | answer = ask_question(st.session_state['transcript'], question) 156 | answer 157 | 158 | -------------------------------------------------------------------------------- /images/api_key.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AssemblyAI-Community/lemur-lecture-summarizer/d2841bc24f83f142011ae7e6a5c1b96499197a51/images/api_key.png -------------------------------------------------------------------------------- /images/file_selection.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AssemblyAI-Community/lemur-lecture-summarizer/d2841bc24f83f142011ae7e6a5c1b96499197a51/images/file_selection.png -------------------------------------------------------------------------------- /images/question_answer.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AssemblyAI-Community/lemur-lecture-summarizer/d2841bc24f83f142011ae7e6a5c1b96499197a51/images/question_answer.png -------------------------------------------------------------------------------- /images/results.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/AssemblyAI-Community/lemur-lecture-summarizer/d2841bc24f83f142011ae7e6a5c1b96499197a51/images/results.png -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | assemblyai==0.17.0 2 | streamlit==1.25.0 3 | python-dotenv==1.0.0 4 | yt-dlp==2023.7.6 -------------------------------------------------------------------------------- /utils/__init__.py: -------------------------------------------------------------------------------- 1 | import streamlit as st 2 | import assemblyai as aai 3 | from yt_dlp import YoutubeDL 4 | 5 | YTDLP_FNAME = 'tmp.webm' 6 | 7 | # For some reason directly importing the constant did not work 8 | def return_ytdlp_fname(): 9 | return YTDLP_FNAME 10 | 11 | def get_transcript(f, ftype): 12 | transcriber = aai.Transcriber() 13 | 14 | print("entered") 15 | print(ftype) 16 | if ftype == 'YouTube link': 17 | with st.spinner('Downloading video...'): 18 | ydl_opts = {'outtmpl': YTDLP_FNAME} 19 | print("downloading") 20 | with YoutubeDL(ydl_opts) as ydl: 21 | ydl.download([f]) 22 | f = YTDLP_FNAME 23 | print("returning", f) 24 | with st.spinner('Transcribing file...'): 25 | transcript = transcriber.transcribe(f) 26 | if transcript.error: 27 | raise TranscriptionException(transcript.error) 28 | return transcript 29 | 30 | def ask_question(transcript, question): 31 | questions = [ 32 | aai.LemurQuestion(question=question,) 33 | ] 34 | 35 | result = transcript.lemur.question(questions) 36 | 37 | if transcript.error: 38 | raise QuestionException(result.error) 39 | 40 | return result.response[0].answer 41 | 42 | 43 | class TranscriptionException(Exception): 44 | pass 45 | 46 | class QuestionException(Exception): 47 | pass --------------------------------------------------------------------------------