├── .DS_Store ├── .gitignore ├── LICENSE ├── README.md ├── data ├── .DS_Store ├── multiple_choice_qa │ ├── EgoSchema.csv │ ├── IntentQA.csv │ ├── NExT_QA.csv │ ├── STAR.csv │ └── TVQA.csv ├── open_ended_qa │ ├── ActivityNet_QA.csv │ ├── MSRVTT_QA.csv │ ├── MSVD_QA.csv │ └── TGIF_FrameQA.csv └── text_generation_benchmark │ ├── Consistency_QA1.csv │ ├── Consistency_QA2.csv │ ├── Generic_QA.csv │ └── Temporal_QA.csv ├── eval_gpt4v_multiplechoice.py ├── eval_gpt4v_openended.py ├── eval_gpt4v_textgeneration_openended.py ├── eval_llava_multiplechoice.py ├── eval_llava_openended.py ├── eval_llava_textgeneration_openended.py ├── evaluation ├── __init__.py ├── direct_answer_eval.py ├── gpt3_consistency_utils.py └── gpt3_evaluation_utils.py ├── example ├── QzulzuWr0Zw_20_45.mp4 ├── extraction_sample │ ├── .DS_Store │ ├── rlQ2kW-FvMk_66_79 │ │ ├── 1.jpg │ │ ├── 2.jpg │ │ ├── 3.jpg │ │ ├── 4.jpg │ │ ├── 5.jpg │ │ └── 6.jpg │ └── ysTmUTQ5wZE_17_45 │ │ ├── 1.jpg │ │ ├── 2.jpg │ │ ├── 3.jpg │ │ ├── 4.jpg │ │ ├── 5.jpg │ │ └── 6.jpg ├── imagegrid_sample │ ├── QzulzuWr0Zw_20_45.jpg │ └── rlQ2kW-FvMk_66_79.jpg ├── rlQ2kW-FvMk_66_79.mp4 └── ysTmUTQ5wZE_17_45.mp4 ├── model_processor ├── .DS_Store ├── __init__.py ├── base_model_inference.py ├── gpt4_model_processor.py └── llava2_model_processor.py ├── pipeline_processor ├── __init__.py ├── gpt4_pipeline.py ├── llava_pipeline.py └── record.py ├── requirements.txt └── vision_processor ├── __init__.py ├── base_frame_extractor.py ├── base_post_processor.py ├── decorator_processor.py ├── fps_extractor.py ├── fps_gridview_processor.py ├── gridview_generator.py └── video_validation.py /.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/.DS_Store -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | 2 | pipeline_processor/.DS_Store 3 | .DS_Store 4 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright <2024> 2 | 3 | Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 4 | 5 | 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 6 | 7 | 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 8 | 9 | 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. 10 | 11 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 12 | 13 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | 3 |

IG-VLM: An Image Grid Can Be Worth a Video: Zero-shot Video Question Answering Using a VLM

4 |
[Paper URL]
5 | Stimulated by the sophisticated reasoning capabilities of recent Large Language Models(LLMs), a variety of strategies for bridging video modality have been devised. A prominent strategy involves Video Language Models(VideoLMs), which train a learnable interface with video data to connect advanced vision encoders with LLMs. Recently, an alternative strategy has surfaced, employing readily available foundation models, such as VideoLMs and LLMs, across multiple stages for modality bridging. In this study, we introduce a simple yet novel strategy where only a single Vision Language Model (VLM) is utilized. Our starting point is the plain insight that a video comprises a series of images, or frames, interwoven with temporal information. The essence of video comprehension lies in adeptly managing the temporal aspects along with the spatial details of each frame. Initially, we transform a video into a single composite image by arranging multiple frames in a grid layout. The resulting single image is termed as an image grid. This format, while maintaining the appearance of a solitary image, effectively retains temporal information within the grid structure. Therefore, the image grid approach enables direct application of a single high-performance VLM without necessitating any video-data training. Our extensive experimental analysis across ten zero-shot video question answering benchmarks, including five open-ended and five multiple-choice benchmarks, reveals that the proposed Image Grid Vision Language Model (IG-VLM) surpasses the existing methods in nine out of ten benchmarks. 6 | 7 | ## Requirements and Installation 8 | * Pytorch 9 | * transformers 10 | * Install required packages : pip install -r requirements.txt 11 | 12 | 13 | ## Inference and Evaluation 14 | We provide code that enables the reproduction of our experiments with LLaVA v1.6 7b/13b/34b and GPT-4V using the IG-VLM approach. For each VLM, we offer files that facilitate experimentation across various benchmarks:Open-ended Video Question Answering (VQA) with datasets such as MSVD-QA, MSRVTT-QA, ActivityNet-QA, and TGIF-QA, Text Generation Performance VQA for CI, DO, CU, TU, and CO, Multiple-choice VQA including NExT-QA, STAR, TVQA, IntentQA, and EgoSchema. 15 | * To conduct these benchmark experiments, please prepare data download and a QA pair sheet. 16 | * The QA pair sheet should follow the format outlined below and must be converted into a CSV file for use. 17 | ```bash 18 | # for open-ended QA sheet, it should include video_name, question, answer, question_id and question_type(optional) 19 | # for multiple-choice QA sheet, it should include video_name, question, options(a0, a1, a2, .. ), answer, question_id and question_type(optional). 20 | # question_id should be unique. 21 | 22 | # example of multeple-choice QA 23 | | video_name | question_id | question | a0 | a1 | a2 | a3 | a4 | answer | question_type(optional) | 24 | |------------|-------------|-------------------------------------------------------|---------------|-------------|----------|----------------|-----------------|------------|-------------------------| 25 | | 5333075105 | unique1234 | what did the man do after he reached the cameraman? | play with toy |inspect wings| stop |move to the side|pick up something| stop | TN | 26 | ... 27 | ``` 28 | 29 | * For experimenting with LLaVA v1.6 combined with IG-VLM, the following command can be used. Please install the LLaVA code to the execution path. Please make sure to reinstall it every time for reproductions. The llm_size parameter allows the selection among the 7b, 13b, and 34b model configurations: 30 | ```bash 31 | # Open-ended video question answering 32 | python eval_llava_openended.py --path_qa_pair_csv ./data/open_ended_qa/ActivityNet_QA.csv --path_video /data/activitynet/videos/%s.mp4 --path_result ./result_activitynet/ --api_key {api_key} --llm_size 7b 33 | ``` 34 | ```bash 35 | # Text generation performance 36 | python eval_llava_textgeneration_openended.py --path_qa_pair_csv ./data/text_generation_benchmark/Generic_QA.csv --path_video /data/activitynet/videos/%s.mp4 --path_result ./result_textgeneration/ --api_key {api_key} --llm_size 13b 37 | ``` 38 | ```bash 39 | # Multiple-choice VQA 40 | python eval_llava_multiplechoice.py --path_qa_pair_csv ./data/multiple_choice_qa/TVQA.csv --path_video /data/TVQA/videos/%s.mp4 --path_result ./result_tvqa/ --llm_size 34b 41 | ``` 42 | * When conducting experiments with GPT-4V combined with IG-VLM, the process can be initiated using the following command. Please be aware that utilizing the GPT-4 vision API may incur significant costs. 43 | ```bash 44 | # Open-ended video question answering 45 | python eval_gpt4v_openended.py --path_qa_pair_csv ./data/open_ended_qa/MSVD_QA.csv --path_video /data/msvd/videos/%s.avi --path_result ./result_activitynet_gpt4/ --api_key {api_key} 46 | ``` 47 | ```bash 48 | # Text generation performance 49 | python eval_gpt4v_textgeneration_openended.py --path_qa_pair_csv ./data/text_generation_benchmark/Generic_QA.csv --path_video /data/activitynet/videos/%s.mp4 --path_result ./result_textgeneration_gpt4/ --api_key {api_key} 50 | ``` 51 | ```bash 52 | # Multiple-choice VQA 53 | python eval_gpt4v_multiplechoice.py --path_qa_pair_csv ./data/multiple_choice_qa/EgoSchema.csv --path_video /data/EgoSchema/videos/%s.mp4 --path_result ./result_egoschema_gpt4/ --api_key {api_key} 54 | ``` 55 | 56 | -------------------------------------------------------------------------------- /data/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/data/.DS_Store -------------------------------------------------------------------------------- /data/text_generation_benchmark/Consistency_QA1.csv: -------------------------------------------------------------------------------- 1 | ,video_name,question,question_id,answer,question_type 2 | 0,v_p1QGn0IzfW0,What equipment is visible in the gym where the boy is doing his routine?,v_p1QGn0IzfW0_0,There is other equipment visible in the gym like a high bar and still rings.,0 3 | 1,v_UZm05CvtxkA,What is happening in the video?,v_UZm05CvtxkA_0,"The video shows a volleyball game being played on a well-lit indoor court, with spectators celebrating in the background.",0 4 | 2,v_oR8o_PuKS28,What are the men playing on the beach?,v_oR8o_PuKS28_0,The men are playing beach volleyball near the shore during daytime.,0 5 | 3,v_YIq3Tar9x6s,What is the color of the sweatshirt worn by the girl who combs her friend’s hair in the video?,v_YIq3Tar9x6s_0,"The girl combing hair is wearing a black sweatshirt, white top, and blue jeans. The other girl is wearing a black sweatshirt.",0 6 | 4,v_U3-MtKZeUXM,What color jersey is the male athlete wearing when he starts the hammer throw?,v_U3-MtKZeUXM_0,The athlete is wearing a blue and red jersey and white and red sports shoes.,0 7 | 5,v_q_QR5HoROhI,What color ink does the man apply to the woman’s nail before curing it?,v_q_QR5HoROhI_0,The man applies black ink to the woman’s nail using a small brush before curing it and then applies glitter of different colors on the black-painted nail which is then cured under a curing lamp.,0 8 | 6,v_qsEnLQ2UnEA,What is the man carrying in the throwing circle?,v_qsEnLQ2UnEA_0,The man is carrying a hammer attached to a handle through a metal string and he is using it as a throwing equipment.,0 9 | 7,v_ZwK52lXCbhA,What is the man wearing while getting tattooed?,v_ZwK52lXCbhA_0,The man getting tattooed is wearing a red sleeveless shirt.,0 10 | 8,v_Y9d4iPDeigI,What color are the shorts and gloves of the boxer in the right corner?,v_Y9d4iPDeigI_0,The boxer in the right corner is wearing red shorts and red boxing gloves.,0 11 | 9,v_qtKa7Lr3URA,What activity is the woman in the white top and jeans engaged in?,v_qtKa7Lr3URA_0,The woman in white is watching the washerwoman wash clothes with a stone on the ground.,0 12 | 10,v_P7Z4i1gmzjs,What is the boy doing at the gym?,v_P7Z4i1gmzjs_0,The boy is swinging between parallel bars in a well-lit gym with foam mats on the floor under various other equipment.,0 13 | 11,v_ojIhxWrBM0s,What is happening in the first shot of the video?,v_ojIhxWrBM0s_0,"In the first shot of the video, a man is sitting in a barber chair, looking through a magazine while draped in a cape, and surrounded by other customers in a salon.",0 14 | 12,v_w4pO-1-FG5w,What kind of cape is draped around the little boy at the saloon?,v_w4pO-1-FG5w_0,The little boy is wearing a blue cape with pictures of animated characters on it at the saloon.,0 15 | 13,v_vo2M4sY0E38,What is the woman wearing while brushing the horse?,v_vo2M4sY0E38_0,The woman is wearing a black top and jeans.,0 16 | 14,v_WOUkPgHtt4E,What is the boy wearing and what is he doing on the armbar?,v_WOUkPgHtt4E_0,"The boy is wearing a white vest and dark blue shorts and is performing different maneuvers on the armbar in a gymnasium with cream color walls, blue mattresses on the floor, windows on the left side, armbars, balance beams, a small wall clock on the left side, trophies kept on the edge of the window and other children practicing around him.",0 17 | 15,v_RjztFj0Zl40,What game are the teams playing on the basketball court?,v_RjztFj0Zl40_0,The teams are playing dodgeball on the basketball court.,0 18 | 16,v_tSNg5rvDvVc,Where is the man playing the saxophone?,v_tSNg5rvDvVc_0,The man is playing the saxophone in a public place with many people watching his performance from behind and there are many stalls and huge black speakers behind him.,0 19 | 17,v_Y16uL6dYDzQ,What is happening in the video?,v_Y16uL6dYDzQ_0,"The video shows a weightlifting competition in a grassy field, with weightlifters lifting big dumbbells on black-colored mats, while applying powder from black buckets. There is an audience watching, officials and a cameraman standing on the left side of the weightlifters, and a jimmy jib camera recording from above. The weightlifters are wearing grey T-shirts, black/red/white shorts and shoes. One weightlifter lifts the dumbbell successfully, while others struggle and drop it, and the video ends with him raising his hand for the audience and walking off the mat.",0 20 | 18,v_wiaZCl-poRw,What is the man wearing while mowing the lawn?,v_wiaZCl-poRw_0,The man is wearing a black polo shirt and cargo pants.,0 21 | 19,v_qqZLs6j3YII,What are the children wearing while they play the guitars on the stage?,v_qqZLs6j3YII_0,The children on the stage are wearing red and white uniforms.,0 22 | 20,v_IRBrTKrVCxo,What is the man doing in the beginning of the video?,v_IRBrTKrVCxo_0,"In the beginning of the video, the man is getting his shoes shined by a man wearing a blue shirt who is polishing his black shoes with a brush outdoors during the daytime.",0 23 | 21,v_xS5JDBeMxOs,What are the men doing with their shoes in the video?,v_xS5JDBeMxOs_0,The men are adding beer to their shoes and drinking from them while the surrounding people cheer for them.,0 24 | 22,v_-uJnucdW6DY,What is happening in the video?,v_-uJnucdW6DY_0,"A group of kids are playing a game of soccer in the grassy field surrounded by a metal fence with light poles. They are passing the ball to each other, kicking it, and running across the field.",0 25 | 23,v_PzJsoZT2hXk,What is the women are doing in the video?,v_PzJsoZT2hXk_0,"The women are gift-wrapping and exchanging wrapped gifts in a well-lit room, and adding gift cards to the wrapped boxes.",0 26 | 24,v_YAMhMB1jdio,What is the color of the cloth capes worn by the customers in the barbershop?,v_YAMhMB1jdio_0,The customers in the barbershop are wearing white cloth capes.,0 27 | 25,v_EWNqkVaOfbE,What does the news reporter in the field wear while interviewing people for TACK?,v_EWNqkVaOfbE_0,"The news reporter wears a grey full-sleeve top and there are various interviewees wearing a blue jersey, white shirt with black coat and black coat with light blue shirt.",0 28 | 26,v_Pjnuoa4o55c,Who is playing squash in the video?,v_Pjnuoa4o55c_0,Four men are playing squash on a blue walled court with polished wooden floors.,0 29 | 27,v_QOsG3BeQeX0,What is the clothing of the person upside down on the parallel bars?,v_QOsG3BeQeX0_0,The person is wearing a blue shirt and khaki shorts.,0 30 | 28,v_zxpQHLwZCug,What is the man wearing and what is he holding in his left hand?,v_zxpQHLwZCug_0,The man is wearing a blue shirt and jeans and is holding an axe in his left hand while standing over a partially cut log in a yard with plants at the back. There is a fence visible at the back with a tree next to it behind and to the right of the man. A house is visible past the fence.,0 31 | 29,v_uvglcAUnQv0,What is the woman in the middle of the dance floor wearing and what type of shoes is she wearing?,v_uvglcAUnQv0_0,The woman is wearing a pink top and a black skirt with high heels.,0 32 | 30,v_T7YcO-sOLuU,What is the man cleaning the shoes with?,v_T7YcO-sOLuU_0,"The man is cleaning the shoes using a hand brush, water, and JASON MARKK cleaning solution.",0 33 | 31,v_OkvWTZ4XLyM,What is the man doing with the discus?,v_OkvWTZ4XLyM_0,The man is spinning the discus to build momentum before releasing it towards the landing area.,0 34 | 32,v_sPK-sbHIb3w,What is happening in the slow-motion clip at the start of the video?,v_sPK-sbHIb3w_0,The players are doing a drill where they dunk the ball into the hoop with a spin.,0 35 | 33,v_TEgPIueNqTI,What is the woman wearing while getting tattooed in the tattoo parlor?,v_TEgPIueNqTI_0,The woman is wearing a white top while she gets tattooed in the tattoo parlor.,0 36 | 34,v_zTHkqpNFGno,What is the woman wearing while she combs her hair?,v_zTHkqpNFGno_0,The woman is wearing a black top and is in a well-lit room with a white wall behind her. There is a desk on her right. She is leaning forward while combing her hair and laughs at something she reads. The video ends as she places the comb at her side and continues to lean forward.,0 37 | 35,v_vska7f8Zm9s,What time of day is it during the video?,v_vska7f8Zm9s_0,The video takes place during the day and the man in the video is adding fresh firewood to the flame.,0 38 | 36,v_SV3v-nA6hXM,What is the man wearing while standing on the bungee platform?,v_SV3v-nA6hXM_0,The man is wearing a white shirt with a harness on it and a cap while standing on the bungee platform.,0 39 | 37,v_w46wkDledek,What are the two girls wearing as they dance?,v_w46wkDledek_0,"One girl is wearing a grey hoodie and pants, while the other is wearing a white and blue shirt and jeans.",0 40 | 38,v_TB5AeIdAndA,What is the man wearing while playing the guitar?,v_TB5AeIdAndA_0,The man is wearing a blue jacket and jeans while playing the guitar on the sidewalk.,0 41 | 39,v_sbnF7VVeopM,What activity is the person in the chef uniform performing?,v_sbnF7VVeopM_0,The person in the chef uniform is washing dishes over a well-lit counter using steel wool and handheld kitchen faucet sprayer to rinse them.,0 42 | 40,v_OESjoURo19U,What game are the kids playing on the basketball court?,v_OESjoURo19U_0,The kids are playing dodgeball on an indoor basketball court.,0 43 | 41,v_SLHPvKh-_x8,What is the woman wearing in the video?,v_SLHPvKh-_x8_0,The woman in the video is wearing a black top.,0 44 | 42,v_ypfWA4aDL6k,Where is the man playing the bagpipe located?,v_ypfWA4aDL6k_0,"The man wearing a kilt is playing the bagpipe on the sidewalk in front of a lamp post. Vehicles are passing behind him on the street, buildings are visible in the background, and people drop change in front of him as they walk past.",0 45 | 43,v_rpjxjU-mLiw,What is the man wearing and what is he doing?,v_rpjxjU-mLiw_0,The man is wearing a teal T-shirt and black shorts and is starting his discus throw in a throwing circle surrounded by a practice net. He then walks towards the camera and claps his hands. A person in a yellow shirt can be seen jogging on the track behind the practice net.,0 46 | 44,v_sk_pnoqXxhs,What is happening in the video?,v_sk_pnoqXxhs_0,"The video shows a toddler in a diaper brushing his teeth with the help of a woman in a white shirt, while a man in a black T-shirt films. There is also a brown dog in the bathroom.",0 47 | 45,v_YK9BIL9fKA0,What are the clothes that the girl recording the video is wearing?,v_YK9BIL9fKA0_0,"The girl recording the video is wearing a maroon T-shirt, blue jeans, and red and white sneakers.",0 48 | 46,v_GvMep9y7d7c,What is the man doing to the cycle?,v_GvMep9y7d7c_0,The man is repairing the chain bearing of the cycle by tightening its screws and hitting it with a branch and a stone.,0 49 | 47,v_ZSdnDVGXU3k,What are the gymnasts performing on the balance beam in the video montage?,v_ZSdnDVGXU3k_0,The gymnasts are performing various gymnastics movements on the balance beam in the video montage.,0 50 | 48,v_u3XOLSC7lg0,What is the man wearing while breakdancing in the hallway?,v_u3XOLSC7lg0_0,"The man is wearing a black T-shirt, black pants, and black and white sneakers.",0 51 | 49,v_skk-nX55gvA,"Who is the person identified as ""Justin Harrison"" throwing the ball towards the goalpost?",v_skk-nX55gvA_0,"Justin Harrison, identified as the person in the white cap, successfully throws the ball towards the goalpost and scores a goal.",0 52 | 50,v_fh21bbDSVmA,What is the color of the gymnast's track pants?,v_fh21bbDSVmA_0,The gymnast is wearing blue track pants and a white vest with blue and red stripes.,0 53 | 51,v_wCexKIuAj3A,What is placed behind the sofa in the news studio?,v_wCexKIuAj3A_0,"There is a TV screen with the word 'Daybreak' behind the sofa, a white coffee table in the middle with coffee cups and a laptop, and five people seated on a sofa in a well-lit studio with a glass window at the back.",0 54 | 52,v_tF8Nv-QxbIU,What color were the suede shoes that the man cleaned?,v_tF8Nv-QxbIU_0,The man cleaned a pair of red suede shoes and a dark blue suede high-heeled shoe.,0 55 | 53,v_vAHR3iJhBXU,What nail polish color did the woman apply on her ring finger?,v_vAHR3iJhBXU_0,The woman applied pink nail polish to her ring finger.,0 56 | 54,v_tC5eMgePakk,What is the little girl wearing and where is she playing the violin?,v_tC5eMgePakk_0,The girl is wearing a pink outfit and pink headdress while playing the violin in front of a tree by the shore of a lake during the daytime. The trees surrounding the lake are reflected in the water behind her. ,0 57 | 55,v_UtV69uNmzlE,What is the woman doing in the well-lit bathroom?,v_UtV69uNmzlE_0,The woman is shaving her right leg with a razor and dips it into the toilet bowl during the process.,0 58 | 56,v_vMcXLYBcHsM,Who gives the man in the video a haircut?,v_vMcXLYBcHsM_0,A woman wearing a hoodie gives the man in the orange t-shirt a haircut.,0 59 | 57,v_u7THn6D6y5w,What is the little boy doing in the gymnasium?,v_u7THn6D6y5w_0,"The little boy is practicing on the armbar with his coach in a well-lit gymnasium full of sports equipment and blue mattresses laid on the floor. His coach is wearing a black T-shirt, black track pants, and black sports shoes and there are posters on the wall. Other children are also practicing in the gymnasium, and some are sitting on the boy’s right side. The video ends with his coach coming to the boy and waving both his hands.",0 60 | 58,v_yggwFa68Cp4,What is the boy wearing while playing the bagpipes?,v_yggwFa68Cp4_0,"The boy is wearing a white shirt, a sleeveless black sweater, and brown pants while playing the bagpipes.",0 61 | 59,v_PwTwGnAVFhQ,What is the woman wearing in the video?,v_PwTwGnAVFhQ_0,The woman is wearing a pink top and black shorts while practicing stepping on and off an aerobics step in a well-lit studio with white walls and a wood floor.,0 62 | 60,v_QT4d1nTQYSE,What is the name of the artist performing a live painting event?,v_QT4d1nTQYSE_0,The artist's name is Jonas Gerard.,0 63 | 61,v_Xmp_twSbkAg,What is the girl wearing while talking in the video?,v_Xmp_twSbkAg_0,"The girl is wearing a black T-shirt and a pink full-sleeve top, and the room is well-lit with white walls, a brown door, a red curtain, some boards on the wall, a small blue table, a desktop, a photo, some books, stickers on the left wall, and a stuffed toy on the study table.",0 64 | 62,v_zDdD_qA86rg,What is the climber wearing?,v_zDdD_qA86rg_0,The climber is wearing a white t-shirt and has a rope attached to their utility belt.,0 65 | 63,v_tt-uDaZQWWc,Who is getting their leg shaved in the video?,v_tt-uDaZQWWc_0,Another woman is having her leg shaved in the video while sitting beside the woman with yellow T-Shirt and multi-colored pajamas. ,0 66 | 64,v_r6l4mxphtX0,What activity is the woman engaged in?,v_r6l4mxphtX0_0,The woman is applying makeup in front of the mirror.,0 67 | 65,v_Oskk7eLgerM,What is the man doing in the video?,v_Oskk7eLgerM_0,The man is giving the dog a bath using a handheld showerhead and soap. He then towels the dog dry and the video ends with a picture of the dog on a blue towel.,0 68 | 66,v_T35cHr4pjkc,What is the man doing in the cafe?,v_T35cHr4pjkc_0,The man is drinking from a cup and later seen dancing in the cafe with another cup in his left hand.,0 69 | 67,v_Pr5K2Jh2X94,What is the color of the athlete’s outfit?,v_Pr5K2Jh2X94_0,The athlete is wearing a red outfit.,0 70 | 68,v_YU-dUWGOoD4,What kind of apparatus is the gymnast getting ready to perform on?,v_YU-dUWGOoD4_0,The gymnast is getting ready to perform on a pommel horse.,0 71 | 69,v_zRqJZ0u6f88,What is the man wearing?,v_zRqJZ0u6f88_0,The man is wearing a red shirt and trousers along with a tool belt.,0 72 | 70,v_QY3q4DuiGq8,What event is the girl participating in?,v_QY3q4DuiGq8_0,The girl is participating in a discus throwing competition and is standing inside the practice net preparing to throw.,0 73 | 71,v_X9Y-b-SbkIU,What is the color of the T-shirt worn by the woman playing squash?,v_X9Y-b-SbkIU_0,The woman is wearing a light green T-shirt and the man is wearing grey shorts.,0 74 | 72,v_uub0z8wJfhU,What interrupts the man while he is playing the violin?,v_uub0z8wJfhU_0,"The man is interrupted while playing the violin, then continues playing until he stops and takes a bow.",0 75 | 73,v_X5kj4fZ38rc,What is the woman wearing while playing the violin?,v_X5kj4fZ38rc_0,"The woman is wearing a dark blue printed top while playing the violin in a well-lit room with white walls, books, photo frames, and a black table on her right side. There is a dining table set and a chandelier in the other room along with a kitchen having a fridge and a microwave with brown wooden shelves.",0 76 | 74,v_rGOOlcdpfLg,What is the woman wearing in the video?,v_rGOOlcdpfLg_0,The woman is wearing a red and blue checkered shirt over a black top.,0 77 | 75,v_fMVuGUBs2cs,What is the name of the cocktail that the bartender is making in the video?,v_fMVuGUBs2cs_0,"The bartender in the video is making a Bloody Mary drink with ice, various ingredients, and garnish.",0 78 | 76,v_PSB1nM3QXxg,What are the boys in the video wearing?,v_PSB1nM3QXxg_0,The boys are wearing white martial arts uniforms.,0 79 | 77,v_TKgU9QJXw2w,What happens to one of the boys during the game in the video?,v_TKgU9QJXw2w_0,"One of the boys accidentally hits another boy in the head while playing racquetball, causing him to fall to the ground and hold his head.",0 80 | 78,v_WAvz9zHwWo4,What kind of competition is the girl participating in?,v_WAvz9zHwWo4_0,The girl is participating in a gymnastics competition and she swings from a high bar.,0 81 | 79,v_uIcMgjypbqk,What happens on the stage at the end of the video?,v_uIcMgjypbqk_0,The video ends with the audience applauding and the man walking off the stage.,0 82 | 80,v_vaV6_bxiTLI,What is the girl wearing?,v_vaV6_bxiTLI_0,The girl is wearing a white top and black shorts.,0 83 | 81,v_z-EPAHsmTMA,What is the advertisement on the running TV about?,v_z-EPAHsmTMA_0,"The advertisement on the TV shows a girl wearing white outfit applying white foam on her face with both hands, washing it off carefully with water later.",0 84 | 82,v_pJ3sECWr5Xg,What are the color of the dresses worn by ballet dancers at the start of the video?,v_pJ3sECWr5Xg_0,The girl is wearing a white ballet dress and the other ballet dancers are wearing black dresses.,0 85 | 83,v_vI-O6QWIIGU,What does the man do after he misses a hold in his gymnastics routine?,v_vI-O6QWIIGU_0,The man applies chalk to his hands after missing a hold in his routine.,0 86 | 84,v_XPiMnbCr-sc,What does the woman do with the peach-colored cloth?,v_XPiMnbCr-sc_0,The woman uses the peach-colored cloth to gift-wrap a box by folding it and tying a knot. ,0 87 | 85,v_w_CXbraJr58,What musical instrument is the man playing in the walkway?,v_w_CXbraJr58_0,The man wearing a Scottish kilt and beret is playing the bagpipe in a walkway.,0 88 | 86,v_ZLJ2BmBkLJg,What type of trimmer did the man use to shave his neck?,v_ZLJ2BmBkLJg_0,The man used two different trimmers and switched between them during his grooming process.,0 89 | 87,v_Sf27WmcLb2w,Who starts the exchange on the volleyball court by serving the ball?,v_Sf27WmcLb2w_0,A man wearing a black sleeveless t-shirt and red shorts starts the exchange on the volleyball court by serving the ball.,0 90 | 88,v_vSv1qMMt4Bk,Where is the reporter Jocelyn Maminta reporting from?,v_vSv1qMMt4Bk_0,Jocelyn Maminta is reporting from an outdoor location during the nighttime. There are cars parked behind her.,0 91 | 89,v_p9lhwYsBWdc,What are the window cleaners wearing as they work?,v_p9lhwYsBWdc_0,The window cleaners are wearing costumes of the Hulk and Batman.,0 92 | 90,v_ZrSK-fisXGg,What is the woman sitting on the chair doing?,v_ZrSK-fisXGg_0,A woman in a store is getting her ears pierced by another woman wearing transparent gloves. The piercer uses an ear-piercing gun to attach a stud to the woman's earlobe and secures it from behind her ears. The video ends with close-up shots of the pierced ears.,0 93 | 91,v_p07Y--q4JQw,Who is applying the clear solution to the woman's nails?,v_p07Y--q4JQw_0,The man wearing a camouflage pattern shirt is applying the clear solution to the woman's nails.,0 94 | 92,v_TsHek6oZYXU,What is the color of the Tai Chi uniform worn by the master?,v_TsHek6oZYXU_0,The master is wearing a black Tai Chi uniform and the other man is wearing a white Tai Chi uniform.,0 95 | 93,v_QN-4aLiDhiQ,What is the color of the shirt that the girl is wearing while getting a tattoo?,v_QN-4aLiDhiQ_0,The girl is wearing a blue shirt over a light blue top while getting the tattoo.,0 96 | 94,v_UGWOtDmIcns,What is the man wearing while rock climbing?,v_UGWOtDmIcns_0,The man is wearing dark blue jeans and black & red shoes while rock climbing.,0 97 | 95,v_PNuJUYo3Hf8,What color wetsuit is the man wearing on the boat?,v_PNuJUYo3Hf8_0,The man on the boat is wearing a blue wetsuit and beanie.,0 98 | 96,v_TMAXjnsN0Mk,What color are the walls in the room where the woman is cleaning the Uggs?,v_TMAXjnsN0Mk_0,The room where the woman is cleaning the Uggs has pink walls.,0 99 | 97,v_skr8x0kUVwg,What are the people wearing in the video?,v_skr8x0kUVwg_0,The people in the video are wearing orange and red T-shirts.,0 100 | 98,v_YnjITZnk6OE,What is happening in the hockey rink during the daytime?,v_YnjITZnk6OE_0,"The players on the hockey rink are wearing full gear and doing drills, which include running side to side, diving, and jumping through obstacles placed on the ground while holding a hockey stick in one hand and wearing hockey gloves in the other. There are brown buildings in the back with trees around them.",0 101 | 99,v_UNJTM62HyqQ,What is the chef wearing in the video?,v_UNJTM62HyqQ_0,"The chef is wearing a white chef's coat, and the man standing behind him is wearing a red T-shirt.",0 102 | 100,v_SEVVSei-r6w,What equipment is the man holding at the start of the video?,v_SEVVSei-r6w_0,The man is holding a hammer with a handle attached to a string and he is practicing hammer throwing by swinging the hammer around him a few times before spinning and throwing it through the open end of the practice net.,0 103 | 101,v_UaO7bS5Ky6M,What is the attire of the woman performing ballet in the theatre?,v_UaO7bS5Ky6M_0,"The woman is wearing a maroon top, black trousers, pointe shoes, and is performing ballet in a well-lit theatre with huge speakers mounted on both sides of the stage, and chandeliers and lights in the galleries.",0 104 | 102,v_wEehiYq9ttE,What is the woman doing as her shoe is being cleaned?,v_wEehiYq9ttE_0,The woman is dancing as her shoe is being cleaned.,0 105 | 103,v_rklYNAAaI3Y,What are the woman and the boy doing in the well-lit room?,v_rklYNAAaI3Y_0,The woman is washing clothes while the little boy tries to help her by pulling clothes from the bucket in the well-lit room.,0 106 | 104,v_ZVjLscGMOs8,What are the women wearing in the video while working out?,v_ZVjLscGMOs8_0,The women are wearing white vests and track pants while working out on aerobics steps in a dimly lit studio with black floor.,0 107 | 105,v_z6pmp8TrAVo,What is the man wearing while washing a plate in the kitchen?,v_z6pmp8TrAVo_0,"The man is wearing a grey T-shirt and glasses, and he is washing a plate in a well-lit kitchen with white walls. There is a refrigerator visible at the back, and a potted plant next to the sink.",0 108 | 106,v_vzUeFzhVYLg,What is happening in the beginning of the video?,v_vzUeFzhVYLg_0,The video starts with a woman on the second level of a diving platform in an indoor swimming pool with overhead lights. The woman runs to the edge and dives with a tuck and several turns into the pool.,0 109 | 107,v_wFVukrQfQhI,What game are the two girls playing in the well-lit court wearing white T-shirts and shorts?,v_wFVukrQfQhI_0,"The two girls are playing racquetball wearing white T-shirts and shorts, and both are wearing safety glasses while playing.",0 110 | 108,v_rOtmhurweqo,What is the color of the old man's T-shirt?,v_rOtmhurweqo_0,The old man is wearing a white T-shirt and track pants.,0 111 | 109,v_RXT17X7lRoQ,What are the colors of the uniforms worn by the girls on the lacrosse team in the video?,v_RXT17X7lRoQ_0,The girls on the lacrosse team are wearing black and pink uniforms.,0 112 | 110,v_YOk1cMsyk88,What is the color of the cap the skateboarder is wearing?,v_YOk1cMsyk88_0,"The skateboarder is wearing a white t-shirt, a red cap, and a backpack.",0 113 | 111,v_qmHjHk0Hzz8,What is the man wearing during the discus throw?,v_qmHjHk0Hzz8_0,The man is wearing black shorts and is shirtless during the discus throw.,0 114 | 112,v_rc-DiL5er28,What are the colours of the t-shirts worn by the dodgeball teams?,v_rc-DiL5er28_0,The dodgeball team on the right is wearing blue t-shirts while the team on the left is wearing yellow t-shirts.,0 115 | 113,v_fM5egP_1DvQ,What color is the vest worn by the first participant in the high jump competition?,v_fM5egP_1DvQ_0,"The first participant in the high jump competition wears a red and green vest, black shorts, and white sports shoes.",0 116 | 114,v_R_JKmGmURvA,What ingredients did the man use to make the cocktail?,v_R_JKmGmURvA_0,"The man used gin, other liquors, and fresh-squeezed lemon juice to make the cocktail. He mixed the ingredients in a cocktail mixer with ice, shook it, and poured it into cocktail glasses.",0 117 | 115,v_vIQPDP8jW8A,What ingredients are used in the cocktail prepared by the bartender in the video?,v_vIQPDP8jW8A_0,"The cocktail shown in the video is made using Bombay Sapphire Gin, St. Germain liquor, lemon juice, ginger syrup, elderflower syrup, slices of cucumber, egg whites, and ice cubes. The bartender crushes the cucumber using a wooden pestle, shakes the mixture with a cocktail mixer, and completes it by adding ice cubes and a slice of cucumber on a straw.",0 118 | 116,v_-pkfcMUIEMo,What is the man wearing and what is he holding in his hand?,v_-pkfcMUIEMo_0,The man is wearing winter clothing and holding a snow shovel. He is demonstrating how to shovel snow by scooping and throwing it to the side.,0 119 | 117,v_VhdidrZKuTU,What does the man sprinkle on the firewood?,v_VhdidrZKuTU_0,The man sprinkles table salt and white pepper powder on the crumpled pieces of newspaper and firewood before lighting it.,0 120 | 118,v_W4mJsJGa0CY,What happens to the man during his pommel horse routine?,v_W4mJsJGa0CY_0,The man loses balance and slips to the floor during his routine.,0 121 | 119,v_TkhUtx0Eyfw,What is happening in the video at the beginning?,v_TkhUtx0Eyfw_0,A girl is getting her ears pierced by a technician in a well-lit store with jewelry and clothes in the back.,0 122 | 120,v_Q3ZNFGE8PZE,What is the boy wearing while dancing?,v_Q3ZNFGE8PZE_0,"The boy is wearing a sweater over a shirt, khaki trousers, white gloves, and a white cap.",0 123 | 121,v_QriYk3MKSnM,What are the two girls doing in the video?,v_QriYk3MKSnM_0,The two girls are seen combing their hair in a dimly lit living room with a shut window at the back.,0 124 | 122,v_x90zpDHTpjQ,What is the man wearing and where is he standing in the video?,v_x90zpDHTpjQ_0,"The man is wearing a white and black t-shirt, standing in a well-lit gym with wooden walls and windows at the top. He attempts to lift a barbell and eventually walks towards the camera to speak.",0 125 | 123,v_X4J9UBXb0FM,What event is taking place in the video?,v_X4J9UBXb0FM_0,The video shows athletes participating in pole vaulting competitions during both day and night.,0 126 | 124,v_zXHyFBrj5Ag,What color is the t-shirt of the man ironing in the video?,v_zXHyFBrj5Ag_0,"The man in the video is wearing a striped blue and black t-shirt and shorts, and is ironing a blue and white Hawaiian shirt on an ironing board in a room with visible clothing behind him.",0 127 | 125,v_PwRqqR1YmPM,What is happening in the video?,v_PwRqqR1YmPM_0,"The video shows athletes competing in a pole vaulting competition in a track field during the daytime, with replays and observers using binoculars.",0 128 | 126,v_peiF2E05u8g,What activities did the girl engage in at the mall?,v_peiF2E05u8g_0,"The girl browsed makeup at a shop, got her makeup done at the salon, got her nails painted, and spoke to the camera at various locations including outside the mall and near an escalator.",0 129 | 127,v_GChGC-VJZHs,What is happening in the video footage?,v_GChGC-VJZHs_0,The video shows dirtbikes performing stunts and jumps on a dirt bike track in a forest. The motorcyclists are equipped with off-road gear and helmets.,0 130 | 128,v_Ur_ToogEkCA,What is happening in the video?,v_Ur_ToogEkCA_0,"In the video, people are playing volleyball during the daytime with thick clouds visible in the sky. There is a woman wearing a blue vest over a red top on the sideline. The game continues with players passing the ball to each other until it ends and a few people enter the frame.",0 131 | 129,v_YAKXrBinGD4,What type of boats can be seen in the video?,v_YAKXrBinGD4_0,"The video shows a houseboat, a jet-ski, a motorboat, and several kayak boats under a shed.",0 132 | 130,v_qU4GmCN2e1Y,What is the man wearing who is performing on the parallel bars?,v_qU4GmCN2e1Y_0,The man performing on the parallel bars is wearing a dark blue vest and pants.,0 133 | 131,v_FqYhC6_CMfI,What is the color of the vest that the man is wearing while windsurfing?,v_FqYhC6_CMfI_0,The man is wearing a dark blue vest and white shorts while windsurfing.,0 134 | 132,v_s9PkkDEl6aY,What is the color of the floor in the basketball court where the Zumba class is conducted?,v_s9PkkDEl6aY_0,"The court floor is blue, and the walls are white and brown.",0 135 | 133,v_ROrLMf0zXko,What is happening on the street in the video?,v_ROrLMf0zXko_0,"On the street during the daytime, there are people standing around a black and white mat as a man in a white outfit is breakdancing. Stores are also lining the sidewalk.",0 136 | 134,v_W6fPv5eoXC4,What is the man wearing on the tennis field?,v_W6fPv5eoXC4_0,The man is wearing a grey sleeveless shirt and blue shorts on the tennis field.,0 137 | 135,v_zB8knKX0W8Q,What type of store is featured in the beginning of the video?,v_zB8knKX0W8Q_0,The video begins with a projecting antique store sign.,0 138 | 136,v_OlykXdYeQOI,What is the barber using to cut the boy’s hair?,v_OlykXdYeQOI_0,The barber is using scissors and a trimmer to cut the boy’s hair.,0 139 | 137,v_PdwWFueQErM,What time of the day was it when the man was shoveling snow?,v_PdwWFueQErM_0,The man was shoveling snow in the daytime and he was wearing a black sweater and jeans.,0 140 | 138,v_TZOhIMjwEE8,What is the woman wearing while lawn-mowing?,v_TZOhIMjwEE8_0,"The woman is wearing a white vest, grey trousers, and white sports shoes while lawn-mowing.",0 141 | 139,v_GH0Umpw6tME,What type of workout is the group of women doing in the video?,v_GH0Umpw6tME_0,The group of women is doing an aerobic workout using aerobic steps.,0 142 | 140,v_UvOEuhS0V3E,What is the woman in the first clip wearing and what is she doing?,v_UvOEuhS0V3E_0,"The woman in the first clip is wearing a white top and applying lipstick in a well-lit room in front of a mirror. The man in the next clip is sitting in front of the same mirror, wearing an olive T-shirt. He removes something from a box and wipes them on his eyebrows one at a time, before eating them.",0 143 | 141,v_ZY0cdXr_1MA,What is happening in the beginning of the video?,v_ZY0cdXr_1MA_0,The video starts with a woman doing ballet in a white room with sunlight streaming from the right.,0 144 | 142,v_QOaPQpXemCA,Where does the video start and what is featured at the beginning of the video?,v_QOaPQpXemCA_0,The video starts with a graphic of a web address for mDhil.com.,0 145 | 143,v_ZK4Y6nOton8,What activity can be seen in the video?,v_ZK4Y6nOton8_0,They are playing badminton in the well-lit venue.,0 146 | 144,v_uCvjzcVCtEs,What is the color of the vest worn by the male athlete in the video?,v_uCvjzcVCtEs_0,"The male athlete is wearing a blue vest, black shorts, and red sports shoes.",0 147 | 145,v_zoWqv-sIdBY,Who is the boy with long hair in the video and where is he sitting?,v_zoWqv-sIdBY_0,"The boy's name is Ivan and he is sitting in a well-lit bedroom in front of a table with a computer on it. He is having his hair combed by Oscar, who is standing behind him.",0 148 | 146,v_qmfZw9eEpIU,What is the color of the windsurfing board the man is using?,v_qmfZw9eEpIU_0,The man is windsurfing on a red board with an attached sail.,0 149 | 147,v_tLk0Y04OaCA,Who is standing next to the girl wearing the blue shirt?,v_tLk0Y04OaCA_0,"The girl wearing the blue shirt is standing next to a person wearing a grey sweatshirt, and another person wearing a red top is visible in the mirror, holding up a phone and recording.",0 150 | 148,v_VWGV8I5byKI,What equipment does the man lift in the gym?,v_VWGV8I5byKI_0,The man picks up a barbell with weights stacked on either end and lifts it once before dropping it to the ground.,0 151 | 149,v_sNJ6_N0j9Ts,What are the men wearing while playing squash in the well-lit court?,v_sNJ6_N0j9Ts_0,The men are wearing shorts and sports shoes.,0 152 | 150,v_t13MQSYjlas,What is the man holding in his hands while standing in the throwing circle?,v_t13MQSYjlas_0,The man is holding a hammer connected to a handle while practicing inside the throwing circle.,0 153 | 151,v_syfyM1Pble8,What color is the T-shirt of the woman who walks away from the exercise bikes?,v_syfyM1Pble8_0,The woman who walks away from the exercise bikes is wearing a grey T-shirt.,0 154 | 152,v_u0A5NIUKEQo,What are the two boys dressed as and what are they doing in the mall?,v_u0A5NIUKEQo_0,"The two boys are dressed as Santa and an Elf and are dancing in the mall. They do backflips, twists, and turns on the floor.",0 155 | 153,v_-vnSFKJNB94,What is happening in the video?,v_-vnSFKJNB94_0,"The video shows a man named Michal Navratil performing various diving maneuvers including tucks, twists, forwards dives, etc. in an indoor swimming pool with people visible walking around. He ends the video by doing a forward tuck dive.",0 156 | 154,v_Ws12Dt0Ny5Q,What is the attire of the female athlete in the stadium?,v_Ws12Dt0Ny5Q_0,"The female athlete in the stadium is wearing a red crop top, red bikini bottoms, and red sports shoes.",0 157 | 155,v_tY9dmG6IyPo,What sport is the woman athlete competing in?,v_tY9dmG6IyPo_0,The woman athlete is participating in a hammer throwing competition in a stadium with spectators.,0 158 | 156,v_wJk3KOUAwXA,What is the man doing in the woodworking shop?,v_wJk3KOUAwXA_0,"The man is waxing and polishing a wooden table using steel wool, a car buffing wheel, and a wet rag.",0 159 | 157,v_wh4JNeQh9FQ,What is the woman wearing while lying on the retractable bed in the parlor?,v_wh4JNeQh9FQ_0,The woman is wearing a black shirt and lying on a retractable bed with a white sheet in a well-lit parlor while getting a tattoo on her earlobe using a tattoo gun. The technician first cleans her ear and then marks the spot before the tattoo artist begins.,0 160 | 158,v_WdJUMmy2ox8,What are the two men wearing and how are they seated?,v_WdJUMmy2ox8_0,"The man on the left is wearing a black shirt and jeans, while the man on the right is wearing a cap and a red sweater. They are seated on a sofa.",0 161 | 159,v_ooFuSspUuVY,What color is the helmet worn by the girl in the video?,v_ooFuSspUuVY_0,The girl is wearing a white helmet while riding the horse in the video.,0 162 | 160,v_tXUssW4vT3o,What is the color of the belly dancer dress?,v_tXUssW4vT3o_0,The belly dancer is wearing a sparkling turquoise-colored dress.,0 163 | 161,v_TGIQtaINzCY,What type of workout class is shown in the video?,v_TGIQtaINzCY_0,"The video shows a cycling class with people on exercise bikes in a well-lit gym with white walls and a grey floor. The instructor is a woman wearing a black t-shirt and a mic in her ear who is calling out instructions, and a man is seen moving a display tower into the class to demonstrate changing the resistance on the exercise bike.",0 164 | 162,v_ZUl43hNzGjA,What are the students doing in the ballet studio?,v_ZUl43hNzGjA_0,The students are practicing ballet at the studio with a teacher wearing a white t-shirt and pants.,0 165 | 163,v_sBwUwgTjhAU,What is the woman wearing over her red outfit in the video?,v_sBwUwgTjhAU_0,The woman is wearing a sweater over her red outfit and looks into the camera before walking away.,0 166 | 164,v_WCS8dzRT7Xk,What is the woman wearing and where is she standing?,v_WCS8dzRT7Xk_0,The woman is wearing a grey shirt and standing behind an ironing board in a well-lit room with a wood wall behind her.,0 167 | 165,v_wjr5M-6qEWI,Where is the referee sitting in the video?,v_wjr5M-6qEWI_0,"The referee is sitting behind a table at the edge of the court, and there are seats for spectators near the walls.",0 168 | 166,v_GOF5guH_kuI,What is the female gymnast wearing while performing on the balance beam?,v_GOF5guH_kuI_0,The female gymnast is wearing a sparkling red costume while performing on the balance beam.,0 169 | 167,v_uqwf903l5-E,What color are the cheerleading uniforms worn by the group in the video?,v_uqwf903l5-E_0,The cheerleaders in the video are wearing blue uniforms with white shoes.,0 170 | 168,v_zxr6UZKPDh4,What are the different colored life vests the kayakers are wearing?,v_zxr6UZKPDh4_0,The kayakers are wearing red and blue life vests over their shirts while kayaking down the river.,0 171 | 169,v_SNJgJB2PGdI,What is the woman dancing with on her head in the video?,v_SNJgJB2PGdI_0,The woman is dancing with a candelabra on her head.,0 172 | 170,v_Zhx6BYVb64g,What does the boy do after taking selfies with the girl?,v_Zhx6BYVb64g_0,"The boy gets up and leaves, and he is later shown knocking on a door and handing a stuffed animal to a girl.",0 173 | 171,v_oncTwYfvCD8,What is happening in the video?,v_oncTwYfvCD8_0,"The video depicts a man demonstrating various martial arts movements in different locations, such as indoors, outdoors, in front of an audience, and in a well-lit studio.",0 174 | 172,v_yttmx8mcjGw,What is happening in the video?,v_yttmx8mcjGw_0,"The video shows various cricket games being played, including professional matches played in stadiums as well as games played by people at homes and beaches. The video also shows a man rolling up a carpet and a woman in a yellow t-shirt getting ready to bat with others around her fielding or watching. The video ends with a child standing near the cricket stumps and getting ready to bat.",0 175 | 173,v_wcxEkMoOmBk,"Who is introduced first in the video, and what is she wearing?",v_wcxEkMoOmBk_0,"Julie, wearing a green sweater, is introduced first in the video while speaking to the camera in a well-lit hair salon, with a round table-top swivel mirror next to her and another woman seated next to her.",0 176 | 174,v_zEttEkAdHts,What items are shown in the beginning of the video?,v_zEttEkAdHts_0,The video begins with rolls of gift wrappers stacked on top of each other and a stack of one US dollar notes placed on a white surface with text displayed above and below it.,0 177 | 175,v_z3MP0yJBtGg,What is the woman wearing and where is she standing?,v_z3MP0yJBtGg_0,"The woman is wearing a black hoodie and jeans skirt and standing in front of a stone dipper well with ladles. There is water in a tank at the top of the dipper. She is standing on a stone floor with a roof. Pillars are visible some distance to her left. A yard is visible outside the structure, with a fence at the end. A tree is visible to the left of the yard.",0 178 | 176,v_Omvg9UtGOes,What colors are on the cloth used to wrap the box?,v_Omvg9UtGOes_0,The cloth used to wrap the box is pink on the inside and green on the outside.,0 179 | 177,v_oni9K1_MCuw,What are the two people doing in the ballroom?,v_oni9K1_MCuw_0,A man and a woman waltz and perform various steps in a ballroom as an orchestra plays music.,0 180 | 178,v_Rn6_IwckXg4,What are the people in the video smoking?,v_Rn6_IwckXg4_0,The people in the video are smoking e-cigarettes.,0 181 | 179,v_TNqc2yWZztE,What is the woman in the video doing?,v_TNqc2yWZztE_0,"In the video, a woman is gift-wrapping boxes using gift-wrapping paper and tape.",0 182 | 180,v_UNXLfqkwKFc,What colour are the shorts worn by the man in the video?,v_UNXLfqkwKFc_0,The man in the video is wearing blue shorts.,0 183 | 181,v_W9H2qVnIWXs,What kind of dog is being bathed in the video?,v_W9H2qVnIWXs_0,A golden retriever is being bathed in the video.,0 184 | 182,v_ZxPF3s_OLDo,What is the man wearing while climbing the hill?,v_ZxPF3s_OLDo_0,The man is wearing a red t-shirt and a belt with a chalk pouch on it.,0 185 | 183,v_Mf3IyeMF8ug,What does the woman use to wrap the box?,v_Mf3IyeMF8ug_0,The woman uses an orange gift wrap to wrap the box and a folded green cloth to tie a knot around it.,0 186 | 184,v_mNiFmkm5OZA,What does the man holding the pipe from the hookah do with the smoke?,v_mNiFmkm5OZA_0,"The man blows smoke rings and then blows smoke into the glass filled with beer, before drinking from it.",0 187 | 185,v_MbjyQGdTtK8,What is the woman in the video demonstrating?,v_MbjyQGdTtK8_0,"The woman is demonstrating how to make furniture polish by mixing olive oil and lemon juice in a spray bottle, shaking it, and then applying it to the surface of a table.",0 188 | 186,v_Gvc1z9qypFQ,What are the colors of the jerseys worn by the two teams playing hockey?,v_Gvc1z9qypFQ_0,"One team is wearing maroon jerseys with black shorts and numbers on the back, while the other team is wearing red jerseys with black shorts and numbers on the back.",0 189 | 187,v_ObrXN_FjVD8,What is the man on the sailboat doing?,v_ObrXN_FjVD8_0,A man in blue t-shirt and glasses is cranking a lever on a sailboat in the sea. The sailboat with maroon sails can be seen sailing in the open sea with cloudy sky. The boat is tilted to one side.,0 190 | 188,v_N75m1Z4RqbA,What are the colors of the clothes the man and woman are wearing?,v_N75m1Z4RqbA_0,"The man is wearing a black shirt and pants, while the woman is dressed in a black outfit.",0 191 | 189,v_Ocid5pcnSVY,What type of dance are the women performing?,v_Ocid5pcnSVY_0,"The women are performing a contemporary ballet with hops, turns, and some synchronized moves.",0 192 | 190,v_MxvKscpouzY,What color shirts are the players wearing in the table tennis match at the beginning of the video?,v_MxvKscpouzY_0,"The player on the near side is wearing a red t-shirt, while the player on the far end is wearing a blue t-shirt and shorts.",0 193 | 191,v_LzPppxSWYNY,What event is the woman participating in the video?,v_LzPppxSWYNY_0,The woman is participating in a long jump competition in the track field during the daytime.,0 194 | 192,v_mMebSMn5D0E,Where do the man and woman go and what do they do once they are there?,v_mMebSMn5D0E_0,"The man and woman go to a piano store. The man plays the piano while the woman stands next to him. Later, they move to another piano. In the closed restaurant, a woman wearing a white robe comes to stand near the man. At an orchestra, a man in a brown jacket is watching the orchestra. He later walks towards the stage and gestures like a conductor.",0 195 | 193,v_FzqZsQx20jY,What is the color of the gymnast's costume?,v_FzqZsQx20jY_0,The female gymnast on the balance beam is wearing a sparkling dark green and white costume.,0 196 | 194,v_-nl4G-00PtA,What is the man doing in the video?,v_-nl4G-00PtA_0,The man is washing dishes in a kitchen sink.,0 197 | 195,v_NvRH4SoF09c,What is the man wearing and what is he doing in the meadow?,v_NvRH4SoF09c_0,"The man is wearing a black shirt, pants and glasses while playing a flute in a meadow with yellow flowered bushes to his right. The sky is cloudy and there is a dirt path stretching up the incline behind him.",0 198 | 196,v_MidLOhRnpnU,What do the men do after hearing the message from the lighter?,v_MidLOhRnpnU_0,The men step away from the electric lighter after hearing the message.,0 199 | 197,v_N5LZMH9mZVU,What kind of routine does the cheerleading squad perform after entering the arena?,v_N5LZMH9mZVU_0,"The all-female cheerleading squad performs a routine that includes flips, somersaults, basket tosses, and stunts involving fliers, with spotters present near each base. They also perform a dance routine and end by posing lifted up by the bases.",0 200 | 198,v_oD8sGLt2FR0,What time of day is it in the video where the boy is shoveling snow?,v_oD8sGLt2FR0_0,"The boy is seen shirtless, shoveling snow in a driveway during the daytime.",0 201 | 199,v_N1_pNhVqG20,What is the surfer doing in the video?,v_N1_pNhVqG20_0,"The surfer is windsurfing and performing flips, maneuvers, and jumps, including a one-handed burner where he pushes the sail down and jump-twists on his board.",0 202 | 200,v_LuH7I7Rxju8,What caused the laptop to fall?,v_LuH7I7Rxju8_0,The man kicked the laptop while breakdancing causing it to fall.,0 203 | 201,v_NojaHuKBagQ,What is the man doing with the adhesive eraser in the video?,v_NojaHuKBagQ_0,The man is using an adhesive eraser to clean the bottom edges and front of the blue and white Nike shoes he is holding in the video.,0 204 | 202,v_Mk7Yjq1U8OI,What type of court are the women playing on during the official warm-up?,v_Mk7Yjq1U8OI_0,The women are doing an official warm-up on a sandy beach volleyball court.,0 205 | 203,v_F9mk2Dn2tFw,What are the colors of the clothes worn by the children in the video?,v_F9mk2Dn2tFw_0,"The children in the video are wearing white, black, pink, and blue clothing.",0 206 | 204,v_o5F7X-UBETE,What is happening in the video?,v_o5F7X-UBETE_0,The video shows a weightlifting competition with athletes from different countries lifting barbells and dropping them on the ground. The competition is well-lit and there are judges sitting behind desks observing the athletes.,0 207 | 205,v_mShp2gzuPOM,What activity is taking place in the gym in the video?,v_mShp2gzuPOM_0,"The video shows a girl playing badminton in a gym with people standing at the edges, a woman taking pictures near the netting, a scoreboard at the top of the wall, an American flag hung vertically, and retracted bleachers to the left.",0 208 | 206,v_meNgZdu8tIk,What kind of sandwich is being made in the video?,v_meNgZdu8tIk_0,"The sandwich being prepared in the video has boiled potatoes, chopped onions, sliced tomatoes, cheese, bell pepper, butter, and green chutney as its ingredients.",0 209 | 207,v_ERYwCKVUTSQ,What is the woman wearing while holding the saxophone?,v_ERYwCKVUTSQ_0,"The woman is wearing a black shirt and black pants while holding a saxophone in a well-lit room with blue and white walls. There is a music stand with sheet music at the back and to her right, a wooden chair to her left and back, and a whiteboard fixed on the back wall.",0 210 | 208,v_myGBEXiYGOU,What is the subject of the painting?,v_myGBEXiYGOU_0,The woman is painting a peacock.,0 211 | 209,v_ltlJ2sOx4as,Where are the riders in the video located?,v_ltlJ2sOx4as_0,The riders in the video are located on the banks of the Terelj River in Mongolia.,0 212 | 210,v_nywOe9C3ajM,What is happening in the beauty salon?,v_nywOe9C3ajM_0,"The girl is getting her hair cut, eyebrows shaped, and sitting in a chair with hair clips, while another woman under a hooded hair dryer reads a magazine.",0 213 | 211,v_mLFjww64sX8,What is the barber doing while standing on the left of the customer?,v_mLFjww64sX8_0,The barber is using a blade on the customer's head while standing on the left of the customer.,0 214 | 212,v_nDHupsLdar0,What is the color of the wall behind the man playing the harmonica?,v_nDHupsLdar0_0,The wall behind the man is cream-colored and there is a lamp visible at his back.,0 215 | 213,v_lSVj34-DR0o,What is the name of the diver featured in the video?,v_lSVj34-DR0o_0,The name of the diver is Thomas Daley and he executes several dives with multiple backward somersaults.,0 216 | 214,v_lU6DMCif3eE,What is the man wearing while playing the bagpipe in the dimly lit room?,v_lU6DMCif3eE_0,The man is wearing a black T-shirt and playing the bagpipe in a dimly lit room with posters featuring text and art.,0 217 | 215,v_gmbC7D6HfuM,What color T-shirt is the man wearing while dancing?,v_gmbC7D6HfuM_0,"The man is wearing a turquoise T-shirt, blue jeans, and white shoes while dancing with the woman wearing a black top and blue jeans with black shoes.",0 218 | 216,v_MWQ9mb_mB8U,What is the color of the tops that the women on the volleyball court are wearing?,v_MWQ9mb_mB8U_0,The women on the volleyball court are wearing white tops with numbers on them.,0 219 | 217,v_mMm1LfVb8Pg,What color T-shirt is the woman wearing while mowing the lawn?,v_mMm1LfVb8Pg_0,The woman is wearing a green T-shirt.,0 220 | 218,v_Nvf-BZNEGdI,What are the people around the boy doing?,v_Nvf-BZNEGdI_0,"The people around the boy are standing and wearing roller skates, and the skating rink has a well-lit polished wood floor.",0 221 | 219,v_-MbZ-W0AbN0,What is the man wearing in the video?,v_-MbZ-W0AbN0_0,The man is wearing a white lab coat in the video.,0 222 | 220,v_mY7QdIn51S8,What teams are playing in the cricket match shown in the beginning of the video?,v_mY7QdIn51S8_0,Pakistan is batting in the cricket match shown at the beginning of the video.,0 223 | 221,v_nI6l7BP0odw,What musical instrument is the man playing?,v_nI6l7BP0odw_0,The man wearing a Scottish kilt is playing the bagpipe.,0 224 | 222,v_NwlIZ1T93xE,What is the man wearing who is buffing the shoes?,v_NwlIZ1T93xE_0,The shoe shiner is wearing a t-shirt and a hat while buffing the customer's shoe.,0 225 | 223,v_Ntyl5alZXeM,What is happening to the girl in the pink top in the well-lit shop?,v_Ntyl5alZXeM_0,The girl in the pink top is getting her ear pierced by a woman wearing gloves and a dark blue top in a well-lit shop.,0 226 | 224,v_N4qhryre_TE,What kind of competition is taking place in the video?,v_N4qhryre_TE_0,The athletes are participating in an indoor tumbling competition.,0 227 | 225,v_MgjxjBxGrQU,What are the three men in the video doing?,v_MgjxjBxGrQU_0,"The three men in the video are dressed as characters from the Mario Bros game, smoking hookah, dancing, and interacting with each other in a well-lit room with a yellow sheet backdrop and a man with a guitar poster on the wall.",0 228 | 226,v_LvaU_4xbxb0,What type of clothing is the girl wearing who is not wearing a black vest and boxing in the gym?,v_LvaU_4xbxb0_0,One of the girls is wearing a black vest and the other is wearing a grey t-shirt while boxing in the well-lit gym.,0 229 | 227,v_nTY4IaGRFIc,What game are the people playing in the swimming pool?,v_nTY4IaGRFIc_0,The people in the swimming pool are playing water polo.,0 230 | 228,v_QG2xYcN_bl0,What is the boy doing on the basketball court?,v_QG2xYcN_bl0_0,The boy is doing a crisscrossing drill with orange agility cones on a basketball court.,0 231 | 229,v_Lshue6AmrIw,What are the children wearing in the mall?,v_Lshue6AmrIw_0,The little girl is wearing a black outfit and the little boy is wearing a green shirt while they are sitting on a shoe-shine chair at a shoe-shine station in a well-lit mall with a store visible behind them.,0 232 | 230,v_eS-VU_XV1K8,What is the color of the instructor's top?,v_eS-VU_XV1K8_0,"The instructor is wearing a pink top, black track pants, and sports shoes.",0 233 | 231,v_NM7Vye5nxj8,What happens when the boy jumps over the bar?,v_NM7Vye5nxj8_0,Both the boy and the man successfully clear the bar during their jumps.,0 234 | 232,v_ghxiG0Dnrt0,What is the woman wearing while cleaning the sink?,v_ghxiG0Dnrt0_0,The woman is wearing a black apron over a T-shirt and jeans.,0 235 | 233,v_hFi6S_guB7I,What are the women doing at the beginning of the video?,v_hFi6S_guB7I_0,The women start by entering the stage from the right and facing away from the audience with their right hands up.,0 236 | 234,v_18yZR3NDaXM,What sport is being played in the video?,v_18yZR3NDaXM_0,The video shows people playing kickball on a field.,0 237 | 235,v_BjjN_aq_wq4,What is happening in the kitchen sink in the video?,v_BjjN_aq_wq4_0,The woman and the young girl are washing dishes with soap in the kitchen sink.,0 238 | 236,v_DJTvj9lfSWU,What is the man doing with his foot on the rooftop?,v_DJTvj9lfSWU_0,The man is cleaning and shaving his right leg with a razor blade on the rooftop while a woman stands beside him holding a microphone.,0 239 | 237,v_0BtHd6dvm78,What is the man wearing as clothing while washing dishes?,v_0BtHd6dvm78_0,The man is wearing glasses and a white hoodie while washing dishes in the kitchen.,0 240 | 238,v_dKJMP8EmZjw,What color are the sails on the dinghy that is shown after the man in the red wetsuit?,v_dKJMP8EmZjw_0,"The dinghy shown after the man in the red wetsuit has red and white sails, and the other dinghies have white sails.",0 241 | 239,v_kGdxtU3NwWo,What is the man wearing while polishing his shoes?,v_kGdxtU3NwWo_0,The man is wearing a dress shirt and has a shoe placed on a black desk mat. He polishes the shoe with a piece of cloth after applying a clear liquid and shoe polish from a tin and glass vile visible on the mat.,0 242 | 240,v_b4xvXZq5uD8,Who brings out the chocolate cake at the end of the video?,v_b4xvXZq5uD8_0,A man wearing a black coat brings out a chocolate cake at the end of the video and the others sing while the man getting the haircut gets his face smeared with chocolate cake.,0 243 | 241,v_aLb79wtvxoQ,What color is the sink in the video?,v_aLb79wtvxoQ_0,The sink in the video is mint green and has two basins.,0 244 | 242,v_KlgrI3Ngwv0,What is the athlete doing at the start of the video?,v_KlgrI3Ngwv0_0,"At the beginning of the video, the athlete is standing by a pommel horse, wearing a blue outfit, and getting ready to start an artistic gymnastics routine.",0 245 | 243,v_jN-jnIgzd6k,What are the children in the video doing?,v_jN-jnIgzd6k_0,"The children in the video are doing karate drills, including different stances, punch and kick drills, and forward and backward movements.",0 246 | 244,v_araMJB23T1s,What do you see at the bottom when the boy jumps from the bungee platform?,v_araMJB23T1s_0,"When the boy jumps, a stream is visible at the bottom.",0 247 | 245,v_dW6OZV43ulU,What are the steps involved in cleaning the leather sofa in the video?,v_dW6OZV43ulU_0,"The process of cleaning the leather sofa involves spraying foam cleaner on the brush and cleaning the couch, then degreasing the damaged areas by spraying some cleaner on a towel and wiping the surface of the sofa, sanding rough surfaces using a scrub pad, using color refresh liquid and wiping the couch with a piece of sponge, drying the surface of the sofa with a hair drier, and applying UV protection liquid on the sofa.",0 248 | 246,v_JcAAwYpJhm8,What is the little girl washing in the kitchen sink?,v_JcAAwYpJhm8_0,"The little girl is washing dishes in the kitchen sink, including a glass, a spoon, a pot, a plate with Dora the Explorer on it, and a plastic jar.",0 249 | 247,v_aYxni8ohTfU,What teams are playing against each other in the volleyball match?,v_aYxni8ohTfU_0,"Italy and Latvia are playing against each other, with Italy wearing blue vests and Latvia wearing red vests.",0 250 | 248,v_6Xsfwn5M6qs,Who can be seen in the background of the video?,v_6Xsfwn5M6qs_0,"Yes, two people are visible in the background - one wearing a green shirt and the other in white sportswear.",0 251 | 249,v_CecqF_uHeOI,What is the man wearing while cleaning the glass door?,v_CecqF_uHeOI_0,The man is wearing a dark blue shirt and khaki pants and uses a larger and smaller wiper to clean the glass door.,0 252 | 250,v_7-taNK5mW8E,What happens just as the ball lands near the front player in the water polo game?,v_7-taNK5mW8E_0,The front player is pushed into the pool from behind just as the ball lands close to her.,0 253 | 251,v_IcR28Yqt4mY,What activity were the men engaged in at the beginning of the video?,v_IcR28Yqt4mY_0,"The men were preparing for a wood-chopping competition, holding axes and standing on blocks of wood.",0 254 | 252,v_B3xMfP_UOt4,What is the attire of the man preparing for the high jump?,v_B3xMfP_UOt4_0,The man is wearing a green fluorescent vest with the name tag 'UKHOV'.,0 255 | 253,v_4XnXyKYxnQQ,What is happening to the ship in the video?,v_4XnXyKYxnQQ_0,The ship is dangerously tilted left and appears to be about to capsize in the daytime sea. ,0 256 | 254,v_hq48S65dlyo,What type of workout is being performed in the video?,v_hq48S65dlyo_0,The video shows a boxing workout where a woman punches punch mitts and kicks pads while a man punches punch mitts and kicks pads.,0 257 | 255,v_gWbIoFrpwTY,What is the little boy doing at the beginning of the video?,v_gWbIoFrpwTY_0,"The little boy is standing in a parking lot holding a juice box, wearing a black t-shirt, with a white van and two skateboards visible in the background.",0 258 | 256,v_eoSwoeLmeyQ,What color are the lights shining on the dance floor?,v_eoSwoeLmeyQ_0,The lights shining on the dance floor are not described in the given details.,0 259 | 257,v_1d_U_2HIPmY,What object does the man have in his hand?,v_1d_U_2HIPmY_0,The man is swinging a hammer attached to a grip by a cable while standing on a circular platform inside a cage.,0 260 | 258,v_Hz7-mBFGsSs,What is happening to the girl in the salon?,v_Hz7-mBFGsSs_0,The girl is getting her hair cut in the salon while crying.,0 261 | 259,v_B_U7vUtoCKY,Who is the focus of the video as he tries to check the ball from the opposing team?,v_B_U7vUtoCKY_0,"Cole Flowler, wearing a white jersey, is the focus of the video as he attempts to check the ball from the opposing team.",0 262 | 260,v_7pfHxav4b0U,What is the woman doing at the start of the video?,v_7pfHxav4b0U_0,The woman is rowing an orange kayak down a rapid river while wearing a life vest and white helmet.,0 263 | 261,v_htzAYwoAZkg,What is the man wearing and doing in the gym?,v_htzAYwoAZkg_0,"The man is wearing a black polo shirt, speaking to the camera in the gym, while the woman is standing next to him; they are both behind a resistance trainer in the well-lit gym. The woman climbs onto the resistance trainer, starts pedaling, and adjusts the instrument cluster settings, while the man continues speaking into the camera.",0 264 | 262,v_ijrjRNwZjqQ,What activity are Kevin Spacey and Rafael Nadal engaged in on stage?,v_ijrjRNwZjqQ_0,Kevin Spacey and Rafael Nadal play ping pong on stage.,0 265 | 263,v_IhmtmKxEU40,What is the color of the wetsuit worn by the boy windsurfing?,v_IhmtmKxEU40_0,The boy seen windsurfing is wearing an orange and black wetsuit.,0 266 | 264,v_3zpODTqnYLs,What is the toddler doing in the video?,v_3zpODTqnYLs_0,The toddler is washing her face with pool water multiple times while sitting on the pool wall while the man holds onto her.,0 267 | 265,v_1KONAdZGdWU,What is the girl doing in the video?,v_1KONAdZGdWU_0,The girl is shaving her left leg while occasionally talking to the camera person and others in the room.,0 268 | 266,v_DIhyfNJAm-M,What is the color of the floors on the stage of the performance?,v_DIhyfNJAm-M_0,"The performer is wearing a black suit and white shirt, playing his saxophone on a stage with red floors, blue curtains and a staircase in front. The audience can be seen sitting a few feet from the staircase, while backing musicians play various instruments behind the saxophonist. A music stand with transparent top is placed next to the saxophonist, and the video ends with the audience applauding.",0 269 | 267,v_K6QunmuTZOQ,What is the woman wearing and where is she standing?,v_K6QunmuTZOQ_0,"The woman is wearing a grey t-shirt and black track pants, standing in a small gym with other exercise equipment around her. Sunlight is streaming from an open door in front of her. ",0 270 | 268,v_l7qSKJyq6mA,What is the man wearing while ironing the dress shirt?,v_l7qSKJyq6mA_0,The man is wearing a black sweater vest over a grey shirt while ironing the dress shirt.,0 271 | 269,v_2VZumBKJuPs,What is happening in the video?,v_2VZumBKJuPs_0,"In the video, a man is piercing the woman's belly button with a needle piercer, and inserting a jewelry before securing it with a ball. The studio is well-lit and the man has black gloves on. Both of his hands are tattooed and the woman can be seen speaking to the person holding the camera",0 272 | 270,v_7EeO9V2B574,What were the women doing in the video?,v_7EeO9V2B574_0,The women were dancing synchronously on the grass under the shadow of a tree to music.,0 273 | 271,v_IgDBDaIoNAM,What are the characters wearing in the video?,v_IgDBDaIoNAM_0,The man and woman are dressed like characters from the Street Fighter game.,0 274 | 272,v_325CAs4hhq8,What sport are the players in orange uniforms playing on the field?,v_325CAs4hhq8_0,The players in orange uniforms are playing field hockey.,0 275 | 273,v_CTJej0rZMiI,Describe the man in the video and his surroundings.,v_CTJej0rZMiI_0,The man is wearing a grey shirt and is sitting in a dimly lit room with an accordion strapped to him. He is sitting next to a wall with a bookshelf behind him and a door frame visible to his right. There are framed Spanish sayings hung on the wall to his side and behind him. He briefly speaks before starting to play the accordion.,0 276 | 274,v_CTyvQHkAKuY,What is the woman doing when the man is playing the piano?,v_CTyvQHkAKuY_0,The woman starts to sing while the man is playing the piano.,0 277 | 275,v_3M290KZV5lo,What do you see in the gymnasium?,v_3M290KZV5lo_0,"The video shows a gymnastics performance on a balance beam in a well-lit gymnasium with judges, spectators, and a balcony with chairs.",0 278 | 276,v_itXYuf_Cj_Q,What is the man wearing while peeling the banana?,v_itXYuf_Cj_Q_0,The man is wearing a green sweatshirt and sitting on a red chair while peeling the banana.,0 279 | 277,v_JspVuT6rsLA,What is the instrument the boy is playing in the video?,v_JspVuT6rsLA_0,The boy is playing the violin in a well-lit room with two pianos at his back and a woman playing one of the pianos.,0 280 | 278,v_cPIsE2ZCWzM,What is happening in the video?,v_cPIsE2ZCWzM_0,"The video shows a polo game being played in the daytime, with players riding horses and striking the ball with their mallets as spectators watch from the red seats.",0 281 | 279,v_IL8__uSFpqg,What is the man wearing in the video?,v_IL8__uSFpqg_0,The man in the video is wearing a blue polo shirt.,0 282 | 280,v_JguRZtFN-Ck,What is the woman wearing while playing squash?,v_JguRZtFN-Ck_0,The woman is wearing a white t-shirt and shorts while playing squash.,0 283 | 281,v_cFOaL4Ir4hg,What kind of dives are being performed in the video?,v_cFOaL4Ir4hg_0,"Several people are diving off the diving board and performing flips, including backflip dives, front flips, and a horizontal flip dive at the end of the video.",0 284 | 282,v_BdRH3lpopLQ,Which two teams are playing against each other in the volleyball game?,v_BdRH3lpopLQ_0,USC and BYU are the two teams playing against each other in the volleyball game.,0 285 | 283,v_JHKAcpYlxkQ,What activity is the woman doing on the tennis court?,v_JHKAcpYlxkQ_0,The woman is playing tennis on an outdoor court and practicing her serve.,0 286 | 284,v_1xBdquA6dx8,What are the colors of the jerseys worn by the two teams playing hockey?,v_1xBdquA6dx8_0,One team is wearing blue jerseys while the other is wearing green jerseys.,0 287 | 285,v_kdOsmDACtqw,What is the action of the man in the black outfit after the man wearing white t-shirt finishes speaking?,v_kdOsmDACtqw_0,The man in the black outfit steps into a white sheet on the ground and starts to breakdance followed by the man in the white t-shirt who dances and interacts with the crowd.,0 288 | 286,v_kIFHChO_-vg,What is the toddler doing in the beginning of the video?,v_kIFHChO_-vg_0,The toddler is sitting on a vanity table with hair rollers in her hair and surrounded by makeup products and bottles while her mother speaks to her.,0 289 | 287,v_5P9Tbut04xE,What is the color of the shirt worn by the man in the video?,v_5P9Tbut04xE_0,The man in the video is wearing a white shirt.,0 290 | 288,v_7gaC3b5nJqs,What are the women wearing in the video?,v_7gaC3b5nJqs_0,The women are wearing gym clothes.,0 291 | 289,v_15yU0R2MBQM,What color is the toothbrush in the video?,v_15yU0R2MBQM_0,The toothbrush featured in the video is blue.,0 292 | 290,v_kgz7c2kb4Rg,What color is the kayak that the woman is in?,v_kgz7c2kb4Rg_0,"The woman is in a yellow kayak, wearing a life vest and a white cap. The kayak is in a calm river surrounded by green hills and trees on the right. She shows how to use the paddle to turn by doing a reverse and forward sweep with both paddles.",0 293 | 291,v_0k0xNs8Zjb4,What is the girl doing in the video?,v_0k0xNs8Zjb4_0,"The girl is performing various gymnastics moves including backflips, front flips, back tucks, standing back-tucks, back handspring tucks, and a tumbling sequence.",0 294 | 292,v_9pavfCL1HvU,What time of day did the game take place?,v_9pavfCL1HvU_0,The game took place in the daytime.,0 295 | 293,v_i5qTK0mInTc,What is the color of the outfit the woman carrying a flower pot is wearing?,v_i5qTK0mInTc_0,One woman is wearing a blue outfit and the other is wearing a pink outfit.,0 296 | 294,v_EK7qW-oavQY,What sport is being played in the pool?,v_EK7qW-oavQY_0,Water polo is being played in the swimming pool.,0 297 | 295,v_jCxQQ9DMpT0,What is the man wearing who instructs the archer on how to grip the bow?,v_jCxQQ9DMpT0_0,The man who instructs the archer on how to grip the bow is wearing a blue polo shirt and a grey cap.,0 298 | 296,v_4NPVpI9zUpI,What is the name of the first jump Stefan Holm performs?,v_4NPVpI9zUpI_0,Stefan Holm performs the first degree jump in which he runs towards the horizontal bar and successfully jumps over it.,0 299 | 297,v_Ku76zinwWkg,What is the colour of the outfits worn by the girl and the boy on the track?,v_Ku76zinwWkg_0,The girl and the boy are both wearing black outfits while running on the track.,0 300 | 298,v_Hy54nZSDxTk,What color are the hoodies worn over the paintball helmets?,v_Hy54nZSDxTk_0,The people are wearing hoodies over their paintball helmets.,0 301 | 299,v_KApqxr02rcQ,What color uniform does Jessica Skillings wear in the video?,v_KApqxr02rcQ_0,Jessica Skillings wears a dark blue uniform in the video.,0 302 | 300,v_BBKSKU6dGGk,What is the man in the blue jacket doing?,v_BBKSKU6dGGk_0,"The man in the blue jacket is shoveling snow on a sidewalk surrounded by a snow-covered lawn and a cloudy sky. A tree covered in snow is visible next to him, and snow-covered trees are visible down the street. The man slowly clears snow as he walks forward, and the video ends with him still shoveling snow. An edge of a car is visible behind him.",0 303 | 301,v_-HpCLXdtcas,What color are the weights on the barbell?,v_-HpCLXdtcas_0,The barbell in front of the man has multi-colored weights on it.,0 304 | 302,v_9VtxNvLx6zA,What sport is being played in the video?,v_9VtxNvLx6zA_0,The video shows a volleyball game being played in a gym.,0 305 | 303,v_AwgGYaV1lT0,What activity is the man engaged in at the beginning of the video?,v_AwgGYaV1lT0_0,"At the beginning of the video, the man is standing in a wooded area chopping wood with an axe. There are trailers visible in the back along with a truck. A tent is set up some distance in the back from the man. A bike is parked in front of it. A foldable table with chairs is visible at the back and right of the man. ",0 306 | 304,v_0X0u_3xUBPE,What is the color of the Tai Chi uniform the man is wearing?,v_0X0u_3xUBPE_0,The man is wearing a black Tai Chi uniform while practicing on a concrete floor outdoors.,0 307 | 305,v_iABuDhagUm8,What does the man do with the axe and the log?,v_iABuDhagUm8_0,"The man uses the axe to chop a log, initially causing it to crack slightly before several swings eventually break the log apart, while a dog periodically comes near the man.",0 308 | 306,v_jt-Vnap5KP4,What is the old man wearing while playing the harmonica?,v_jt-Vnap5KP4_0,"The old man is wearing winter clothes, a hat, and sunglasses while playing the harmonica.",0 309 | 307,v_agu58IVL7IQ,What is happening in the background while the man is skateboarding?,v_agu58IVL7IQ_0,Other people are visible skating in the background besides the man with the skateboard.,0 310 | 308,v_9AfQ2Q-96oE,What is happening in the video?,v_9AfQ2Q-96oE_0,"The man is participating in a disk throw competition, where he spins to build momentum and throws the disk, with a referee marking the spot where the disk landed. The video shows different angles of the man throwing the disk and several clips of the man making different disk throws one after another. There are also hundreds of spectators visible sitting on seats.",0 311 | 309,v_A904-cbaYDw,What is the man doing in the video?,v_A904-cbaYDw_0,The man is performing a tai chi practice and instructing on different stances and their positions.,0 312 | 310,v__B7rGFDRIww,What song is the woman dancing to in the beginning of the video?,v__B7rGFDRIww_0,"The woman is dancing to the song ""Let it Go"" from the movie ""Frozen"" and this song continues to play throughout the video while the dancers dance to it.",0 313 | 311,v_5tFfEA1JWjA,What is happening in the video?,v_5tFfEA1JWjA_0,A boy is dunking the basketball while his coach watches and instructs him in an indoor basketball court with frames on the far wall.,0 314 | 312,v_AQnp2QAnwI0,What is happening in the video?,v_AQnp2QAnwI0_0,A group of dancers perform on stage in front of a projection while silhouettes of people wearing headphones are visible in front of the stage.,0 315 | 313,v_ANaMmXJFE54,What are the cheerleaders doing before they start their routine?,v_ANaMmXJFE54_0,The cheerleaders run into the basketball court and wave towards the spectators.,0 316 | 314,v_7orO93DICYw,What kind of equipment is in the gym where the man is practicing?,v_7orO93DICYw_0,"The gym where the man is practicing gymnastics has overhead lights, several balance beams fixed to the ground with foam pads lining the floor, and numerous flags lining the wall.",0 317 | 315,v_9hTcvGUnMBs,What activity is taking place in the field at the beginning of the video?,v_9hTcvGUnMBs_0,"At the beginning of the video, a high jump contest is taking place in a field with a horizontal bar, foam pad, people competing, a man with colored flags, and a seated audience.",0 318 | 316,v_60tIsGR0Zzg,What does the boy do before mowing the lawn and what happens when he tries to start it?,v_60tIsGR0Zzg_0,"Before mowing the lawn, the boy switched on the mower and moved it forward. He struggles to move it and gets stuck in some patches, eventually pulling the mower back. ",0 319 | 317,v_IRua6QGP-Ow,What is the bartender doing in the video?,v_IRua6QGP-Ow_0,"The bartender is making a drink by pouring and mixing different spirits, shaking them with ice, and finally pouring the mixture into a glass with ice. He also runs a lime on the rim of the glass.",0 320 | 318,v_DOI6tsATsE4,What is the boy wearing and doing in the video?,v_DOI6tsATsE4_0,The boy is wearing a dark blue sweatshirt and holding his hand to the side of his head while sitting at a table with a plate in front of him. The background shows a lamp near the wall. The man also appears in a cutaway wearing a white t-shirt and looking at the camera while standing in a room with a packed shelf. The video ends with cutaway shots of the boy looking at the camera and looking away.,0 321 | 319,v_ej9TkJVjK9M,What color tights is the man wearing?,v_ej9TkJVjK9M_0,The man is wearing black tights.,0 322 | 320,v_8L1UjGLOCeI,What is the woman doing in the room with the hopscotch court?,v_8L1UjGLOCeI_0,The woman is playing hopscotch in the room and completes a pass on the court.,0 323 | 321,v_c7S423zJqPo,What is the attire of the man in the aerobics class?,v_c7S423zJqPo_0,The man wearing a green t-shirt is working out using an aerobics step in the aerobics class.,0 324 | 322,v_Bkhr72sazvQ,What is the weight loss product being advertised in the video?,v_Bkhr72sazvQ_0,"The man in the video is promoting a weight loss product and talks about ""5 pounds lost in 7 days"".",0 325 | 323,v_d-RYiiPzVKg,What items does the man place on the table in the video?,v_d-RYiiPzVKg_0,The man places a bottle of white vinegar and a bottle of olive oil on the table.,0 326 | 324,v_BmLDHLKiIdQ,What type of competition is being held in the video?,v_BmLDHLKiIdQ_0,The video shows an indoor archery competition with spectators seated in the stands.,0 327 | 325,v_9VflCeoPr9I,What activity are the people engaged in the video?,v_9VflCeoPr9I_0,The people are playing and kicking a red ball in the field.,0 328 | 326,v_1kwIYXB4b8s,What is happening in the video?,v_1kwIYXB4b8s_0,The video shows a man getting his head shaved by a woman while a man records it on his phone in the presence of other people in the room.,0 329 | 327,v_kPn9NzU4W48,What is the color of the basketball court walls behind the two teens?,v_kPn9NzU4W48_0,The basketball court walls behind the two teens are painted white at the top half and blue at the bottom.,0 330 | 328,v_4iFlM_Zu7xs,What are the riders doing in the video?,v_4iFlM_Zu7xs_0,"The riders are performing a series of jumps and stunts on a dirt bike track, while being recorded by a person in the middle. They jump over inclines, ride in a circle and jump over a hill at the end.",0 331 | 329,v_Dbas61NkTV0,What props are being used on the cheer squad's stage?,v_Dbas61NkTV0_0,"The stage has overhead lighting and gobo lighting projections on the wall, and the cheer squad uses multiple bases to throw and lift members during their routine.",0 332 | 330,v_j6ElczD-Pw4,What is the girl doing with the luffa?,v_j6ElczD-Pw4_0,The girl is washing her face with the luffa by tapping it on her face and then rinsing off the suds with water from the running sink.,0 333 | 331,v_e3DB_y78IHc,What are the animals shown grazing in the video at the end?,v_e3DB_y78IHc_0,"The animals shown grazing towards the end of the video are ostriches, alpacas, and camels.",0 334 | 332,v_9ZVWD0ZKXZg,What is the boy wearing while climbing the rock?,v_9ZVWD0ZKXZg_0,"The boy is wearing an orange T-shirt, a harness, and a belay rope is fastened to him.",0 335 | 333,v_1dDAcUliXrQ,What color is the badminton court?,v_1dDAcUliXrQ_0,The badminton court has a dark blue floor and walls.,0 336 | 334,v_KvtqF0LaWU0,What is the woman wearing in the video while standing beside the black stone kitchen counter?,v_KvtqF0LaWU0_0,The woman standing beside the black stone kitchen counter is wearing a sweater.,0 337 | 335,v_3VjaC7wdMso,Where is the girl sitting and what is she doing in the video?,v_3VjaC7wdMso_0,"The girl is sitting by an open window playing the piano while referencing the sheet music in front of her, and she starts to sing as she plays. Trees and sunlight are visible outside the window and the edge of a bed is visible behind her chair.",0 338 | 336,v_cms1BP4eKOA,What are the players doing in the video?,v_cms1BP4eKOA_0,"The players are performing a point guard drill, running around cones, attacking the basket, getting the ball back from the rebounder machine, dribbling while running, and making a point guard entry pass to attack the basket.",0 339 | 337,v_Ivq9_wPFNs4,What event is taking place in the video?,v_Ivq9_wPFNs4_0,"The video depicts a woman named Britney Reese performing a high jump into a sand pit during an athletics event while her stats are displayed on a graphic, and is followed by her running in a lane and people sitting in the seating area.",0 340 | 338,v_dO01vfB9z4g,What is the boy doing in the video?,v_dO01vfB9z4g_0,"The boy is scrubbing dishes in the kitchen sink while standing on a box, and a woman records him with her phone.",0 341 | 339,v_4-KiAur4dI8,What is the name of the event featured in the video?,v_4-KiAur4dI8_0,"The event featured in the video is Red Bull CROSSWAYS, as indicated by the logo displayed at the beginning of the video.",0 342 | 340,v_1XNHcfxTqW0,What is the man wearing while playing the harmonica on the sidewalk?,v_1XNHcfxTqW0_0,The man playing the harmonica on the sidewalk is wearing a white T-shirt and has long hair.,0 343 | 341,v_AE2gHOwcV14,Where is the video shot?,v_AE2gHOwcV14_0,The video is shot in a gym with a visible glass door through a divider wall separating the gym from the rest of the hall.,0 344 | 342,v_bphckj0BV5U,What is the woman wearing while demonstrating how to start a fire?,v_bphckj0BV5U_0,The woman is wearing a grey top and is sitting on the ground in the woods while demonstrating how to start a fire.,0 345 | 343,v_E5YBb4rzV30,What ingredients does the woman add to the mixing bowl before adding tuna?,v_E5YBb4rzV30_0,"Before adding a can of tuna to the mixing bowl, the woman adds mayonnaise, relish, mustard, celery, and chopped onion. She then arranges lettuce leaves on a plate, places a spoonful of the tuna mixture on each leaf and folds them into wraps.",0 346 | 344,v_dRF1C_sBE5M,What was the girl shown doing with the axe in the video?,v_dRF1C_sBE5M_0,The man taught the girl how to chop wood using an axe.,0 347 | 345,v_H33jSILKmfI,What uniform is the boy wearing while practicing training drills in the gym?,v_H33jSILKmfI_0,The boy is wearing a white martial arts uniform while practicing training drills in the gym.,0 348 | 346,v_JiNcVb97CnA,What is the attire of the man and woman in the video?,v_JiNcVb97CnA_0,The man is wearing a white t-shirt and the woman is wearing a black coat over a pink top.,0 349 | 347,v_lIo8HfzzSNs,What ingredients were used to make the dressing for the Russian salad?,v_lIo8HfzzSNs_0,"The dressing for the Russian salad was made by mixing two T-spoons of cream, two T-spoons of mayonnaise, salt, and pepper in a glass bowl.",0 350 | 348,v_dN8LBey17O8,What does the man do before mowing his lawn?,v_dN8LBey17O8_0,The man measures the length of the grass using a ruler before and after mowing his lawn.,0 351 | 349,v_I7slpTgl0jc,What is the man practicing in the field?,v_I7slpTgl0jc_0,The man is practicing a hammer throw in a field surrounded by trees. He is standing on a round platform holding a handle with a hammer attached to it. He is wearing a red shirt and releases the hammer through the open end of the practice net.,0 352 | 350,v_ks018ZYzrx4,What is the color of the dirt bike with the number 2?,v_ks018ZYzrx4_0,The dirt bike with the number 2 is green.,0 353 | 351,v_KsFid_YVsn0,What are the colors of the costumes that the girls and boys wear when they start dancing?,v_KsFid_YVsn0_0,"The girls are wearing black dresses, and the boys are wearing red dress shirts and black trousers while dancing.",0 354 | 352,v_hFpSoRlpyMo,What happens to the ball hit by the batsman towards the boundary?,v_hFpSoRlpyMo_0,The fielder prevents the ball from touching the boundary rope and falls over it.,0 355 | 353,v_aOxRwTlkY80,Which ingredients did the man add to the pan first?,v_aOxRwTlkY80_0,The man added garlic to the pan with oil first and then added chillies. He cooked these two for a minute and later added the cooked pasta and chopped parsley. He added ground pepper and mixed everything before finishing the dish.,0 356 | 354,v_4VdS1eN3dv4,What is the common theme in the different cricket games in the video?,v_4VdS1eN3dv4_0,"Yes, multiple games of cricket are being played in the video, both during the day and at night.",0 357 | 355,v_EhzHU0gkrzA,What is the boy doing in the video?,v_EhzHU0gkrzA_0,"The boy is performing gymnastics on the parallel bar in a track field with blue flooring. He swings, performs a handstand, and repeats this while turning in the other direction. Another person is doing gymnastics on the far right of the field.",0 358 | 356,v_6G8m-vsGTA8,What is happening in the video?,v_6G8m-vsGTA8_0,"The video shows a man getting his hair cut in a hair salon while recording himself, with a woman using scissors and trimmers to cut his hair. Other customers can be seen in the salon and the video ends with the man driving in a car wearing a hat and sunglasses.",0 359 | 357,v_kfO5PGlsMJw,What color is the karate uniform of the first boy who performs on the stage?,v_kfO5PGlsMJw_0,The first boy is wearing a black karate uniform with a black belt.,0 360 | 358,v_86Unahb4sJM,What makeup products did the woman apply in the video?,v_86Unahb4sJM_0,"The woman demonstrated the application of concealer, foundation, bronzer, eyeliner, mascara, lipgloss, and eyebrow mascara in the video.",0 361 | 359,v_jN2CFk0xKp8,What is the man wearing and what is he carrying in the first clip?,v_jN2CFk0xKp8_0,The man is wearing a red polo shirt and blue trousers and carrying a vacuum on his back while vacuuming the carpet in an office.,0 362 | 360,v_JG7T6qtO4jg,What is the boy wearing in the video?,v_JG7T6qtO4jg_0,The boy who starts breakdancing in the beginning of the video is wearing a black T-shirt and olive shorts.,0 363 | 361,v__ja1fre9OTE,What is the apparatus the gymnast performs on?,v__ja1fre9OTE_0,The gymnast is performing on a balance beam located on an elevated platform at a gymnastics competition.,0 364 | 362,v_82EtiBkAsPg,What is the man wearing while playing the bagpipe at the bar?,v_82EtiBkAsPg_0,The man is wearing a black jacket while playing the bagpipe at the bar.,0 365 | 363,v_7OcxT66BxX0,What is the man wearing while he sings on stage?,v_7OcxT66BxX0_0,The man is wearing a black jacket and a black fedora while playing the harmonica on stage.,0 366 | 364,v_aDe5hUB3AdY,What are the sailors seen doing in the video?,v_aDe5hUB3AdY_0,The sailors are performing capsizing training and sailing dinghies.,0 367 | 365,v_5qY9yPt-5Go,What is happening in the beginning of the video?,v_5qY9yPt-5Go_0,"The video starts with a black French bulldog standing on a checkered red tablecloth with a cup of coffee placed in front of it. The dog goes to the cup and starts to drink the coffee, while the boy attempts to move the dog away from the cup gently.",0 368 | 366,v_hvrKRg166eQ,What are the people doing on the couch?,v_hvrKRg166eQ_0,"Three people are sitting on a curved couch behind a coffee table, with three coffee cups on the table. A woman is sitting in the middle with two men on either side of her.",0 369 | 367,v_a_JlhLnJM98,What is the woman wearing and what is behind her?,v_a_JlhLnJM98_0,"The woman is wearing a green sweater and there is a jeans jacket visible hanging behind her. She is standing in a well-lit room with a window behind her. Behind her, to the left, is a rack of different products. ",0 370 | 368,v_9Ufy_JGbnys,What type of dance are the couple engaged in?,v_9Ufy_JGbnys_0,The couple is doing a tango dance.,0 371 | 369,v_9eniCub7u60,What do you see in the back on the right side of the room?,v_9eniCub7u60_0,"Yes, there is a door visible on the back right side of the room and a few photo frames hanging on the wall.",0 372 | 370,v_k1GP0cJOXbQ,What are the different activities people engage in throughout the video?,v_k1GP0cJOXbQ_0,"The video shows people kayaking in rapids and calm waters, with some scenes of them paddling alone and others in pairs, culminating in a group of people standing around drinking beer with the sunset in the background.",0 373 | 371,v_7bwruwic1CI,What game are the players in orange uniforms playing at the beginning of the video?,v_7bwruwic1CI_0,"The players in orange uniforms are celebrating a baseball game, and when the third player falls on the field, handball is being played.",0 374 | 372,v_bYxVbXzQToI,What is the man in the white suit doing?,v_bYxVbXzQToI_0,The man in the white suit sits on a chair on a raised platform in a bar and picks up a guitar with a man in a black tux sitting next to him holding a guitar. They start playing while people in the bar watch and drink.,0 375 | 373,v_hI11n4_r178,What type of outfits are the two women wearing?,v_hI11n4_r178_0,The two women in the video are wearing red shirts.,0 376 | 374,v_hP-xRyx88aA,What is the woman in the white jacket doing in the video?,v_hP-xRyx88aA_0,"The woman in the white jacket is pole vaulting over the bar, using her pole to clear it and landing on the foam mat. This is the second jump she makes in the video.",0 377 | 375,v_HxrtuZSCwvY,What instrument is the boy holding in the video?,v_HxrtuZSCwvY_0,The boy is holding a violin and playing it in a dimly lit living room while people sit and watch. He stops playing at the end and people applaud.,0 378 | 376,v_af05Mqm03Y4,What color T-shirt is the man wearing who is not tired?,v_af05Mqm03Y4_0,One man is wearing a white T-shirt and the other is wearing a grey T-shirt.,0 379 | 377,v_ddzfgHw_qlU,What is the man wearing and what is he doing?,v_ddzfgHw_qlU_0,"The man, who is wearing a grey dress shirt and glasses, is seen playing a harmonica in front of a painting in a well-lit room with a yellow overhead light. He continues to play until the end of the video.",0 380 | 378,v_h_r8gioFi4k,What are the two people wearing and what are they doing in the gym?,v_h_r8gioFi4k_0,"The man is wearing a black outfit and punch mitts while the woman is wearing a black outfit and boxing gloves. They are practicing kickboxing by using various moves including punching, kicking, and using elbows. The gym is well-lit and has posters on the walls while a multicolor mat covers the floor. Through an open door, a hall with chairs and wooden flooring can be seen.",0 381 | 379,v_EnMS9O1U6E0,What event is taking place in the stadium?,v_EnMS9O1U6E0_0,A man wearing blue shorts and a vest with the number tag of 2736 is shown performing a high jump at a track and field event hosted at an open-roof stadium with numerous spectators.,0 382 | 380,v_CeXtKbFnPIQ,What is the man wearing and holding on the tennis court?,v_CeXtKbFnPIQ_0,The man is wearing a black shirt and black track pants and holding a racket and a tennis ball on an outdoor tennis court.,0 383 | 381,v_K_ia4VP3w2c,What is the name of the model in the video?,v_K_ia4VP3w2c_0,Irina Shayk is the model for the makeup tutorial and photoshoot in the video.,0 384 | 382,v_IDIALY6Ayk8,What activity are the people engaged in at the winery?,v_IDIALY6Ayk8_0,"The people are at a winery, where they are shown a bottle of wine and then served wine to drink while seated in a dining room.",0 385 | 383,v_ij9czNGMR5I,What is the woman wearing while getting her tongue pierced?,v_ij9czNGMR5I_0,The woman getting her tongue pierced is wearing a grey hoodie.,0 386 | 384,v_jEOtcfk220s,What sport is being played?,v_jEOtcfk220s_0,A women volleyball match is being played in a light wood-floored gymnasium with white walls and bleachers at the bottom. The players on the left wear blue uniforms and a player makes a successful serve while a referee watches over the net. The video ends as the team on the left wins the exchange.,0 387 | 385,v_j6OcN4OQZVM,What is the color of the outfits worn by the breakdancers in the video?,v_j6OcN4OQZVM_0,The breakdancers in the video are wearing black outfits.,0 388 | 386,v_bz4QwqHiVlo,What is the color of the shirt worn by the man in the living room?,v_bz4QwqHiVlo_0,The man is wearing a purple shirt while vacuuming in the living room.,0 389 | 387,v_hPYGgv_75SY,Where does the woman wearing a black outfit perform the hammer throws?,v_hPYGgv_75SY_0,The woman wearing a black outfit is performing hammer throws on a field in the daytime.,0 390 | 388,v_jWH-2aa1gJY,What is the girl in the video wearing and what is visible behind her?,v_jWH-2aa1gJY_0,The girl in the video is wearing a hoodie and jeans and sitting on a chair with a closed window behind her. There is various music gear visible around her on the ground and the room is lit by diffuse light from the window.,0 391 | 389,v_BAtlo_AGFWs,What is Chris Hadfield doing while holding a tortilla in his hand?,v_BAtlo_AGFWs_0,Chris Hadfield is holding a tortilla and squeezing honey on it while floating in the International Space Station.,0 392 | 390,v_9lAT3oA9Ixs,What is the man wearing while chopping wood in the woods?,v_9lAT3oA9Ixs_0,The man is wearing a blue jacket and blue jeans while chopping wood near a cabin in the woods.,0 393 | 391,v_0rr7iGHamw0,What is Valarie doing in the crowded hall?,v_0rr7iGHamw0_0,"Valarie is using iron-on transfer to imprint a design on a white t-shirt with a Twitter username and blue bird picture, while standing in front of an ironing board holding an iron in the crowded hall.",0 394 | 392,v_6utacRir3uM,What is happening in the aerobics studio?,v_6utacRir3uM_0,People are working out in a well-lit aerobics studio using aerobic steps with an instructor giving instructions. A man falls down while trying to follow the instructor and receives help from the instructor and some other participants.,0 395 | 393,v_5cijPqKVc84,What kinds of kayaks are shown in the video?,v_5cijPqKVc84_0,"The video shows various types of kayaks being rowed in different water bodies including rapids, rivers, and the sea. Some have open cockpits while others are closed.",0 396 | 394,v_l5ZUUChppHU,What type of outdoor competition is taking place in the video?,v_l5ZUUChppHU_0,"The video shows a wood-chopping contest with participants standing over their respective logs and two sets of competitions - wooden blocks laid horizontally on chopping stands and contestants assembling wooden blocks on horizontal chopping stands. They are wearing black vests and light-colored trousers, and the contest takes place outside in an arena during the daytime.",0 397 | 395,v_52Xw6TZcysc,What event is the woman participating in?,v_52Xw6TZcysc_0,The woman is doing a long jump in an indoor track and field stadium.,0 398 | 396,v_l_0HvMt2tGg,What sport is being played in the video?,v_l_0HvMt2tGg_0,"The video features the boy playing lacrosse, wearing a red and white jersey, and using a lacrosse stick.",0 399 | 397,v_1jgsRbGzCls,What is the video mainly about?,v_1jgsRbGzCls_0,"The video is about Tai chi and its practice by people of different ages and in different settings, including a newsroom, sandy ground outdoors, a hospital, and individuals practicing on their own.",0 400 | 398,v_kl-2y3jtPNE,What game is being played in the field?,v_kl-2y3jtPNE_0,The game being played in the field is cricket.,0 401 | 399,v_j_r7zJWxI8w,What is the color of the shirt worn by the man picking up the beer can?,v_j_r7zJWxI8w_0,The man wearing the Adidas blue t-shirt makes a hole in the beer can using a key and drinks from it.,0 402 | 400,v_l_pp-1qu9Ig,What is the woman doing with the gift wrap?,v_l_pp-1qu9Ig_0,"The woman is wrapping a brown cardboard box with gift wrap by cutting, folding and taping it, and then placing it on the counter.",0 403 | 401,v_8r4bF9RF0Kc,What sport are the people playing at the beginning of the video?,v_8r4bF9RF0Kc_0,The people in the video are playing a game of cricket at the beginning.,0 404 | 402,v_iUiXNXRs4rU,What is the man wearing and where is he located?,v_iUiXNXRs4rU_0,"The man is wearing a brown shirt and a cap, standing in a well-lit bike shop with shelves filled with tools and bike parts. He is fixing a broken chain by attaching a master link on one end and pushing on the pedal to check the fix.",0 405 | 403,v_Lf_7RurLgp0,What is the little boy wearing while he washes his hands?,v_Lf_7RurLgp0_0,The little boy is wearing a black polo shirt while washing his hands in the video.,0 406 | 404,v_34ZxyyFjbHQ,What are the colors of the T-shirts of the two kids playing squash?,v_34ZxyyFjbHQ_0,One boy is wearing a blue T-shirt and the other a red T-shirt. The game gets started over when one of them misses their shot. The video ends with the two boys playing and a boy wearing a white t-shirt standing outside the court watching. It doesn’t mention which boy wins the starting-over games.,0 407 | 405,v_Db9VWkgMpbE,What is the woman wearing who is learning to use an exercise bike?,v_Db9VWkgMpbE_0,The woman who is learning to use an exercise bike is wearing a black top.,0 408 | 406,v_Il9Ara3IqeY,What is the man doing on the Marble Bridge during the daytime?,v_Il9Ara3IqeY_0,"The man is standing on an elevated section of Marble Bridge, playing a flute. He is wearing a black polo shirt and a rice hat. The Palace Museum can be seen at his back. People are walking around Meridian Gate Square, some of whom are walking under an umbrella. He continues playing the flute until the end of the video.",0 409 | 407,v_HkbOI7gEtKk,Who are the two women in the video and what are they doing?,v_HkbOI7gEtKk_0,"The video features two women, one wearing a teal top and the other wearing a white one. The woman in the teal top is a beautician and she is demonstrating how to make a cleansing with baking soda while the woman in the white top, the model, follows the instructions by mixing yogurt and baking soda to apply to her face.",0 410 | 408,v_cEa87QoI3Do,What do the two men make in the kitchen?,v_cEa87QoI3Do_0,"The two men make sandwiches using various ingredients like peanut butter, jelly, lettuce, and ham.",0 411 | 409,v_ARpoJIBF2NM,What color are the t-shirts worn by the players on one side of the basketball court?,v_ARpoJIBF2NM_0,The players on one side of the basketball court are wearing teal t-shirts.,0 412 | 410,v_H8oehT8eeGc,What happens to the red ball tossed by a boy?,v_H8oehT8eeGc_0,"The red ball is thrown by a boy, retrieved by another boy wearing a green shirt who then kicks it. The ball flies through the air and lands near the opposite chain-link fence. A boy wearing a red shirt then runs toward the ball.",0 413 | 411,v_jQzBXawu29s,What is the man wearing while playing the black wooden Irish flute in the living room?,v_jQzBXawu29s_0,The man is wearing a brown sweater and glasses while playing the black wooden Irish flute in a living room with a television stand behind him and various framed items hung on the wall. Sunlight is coming in from the left side at the back.,0 414 | 412,v_51faGoLPJgM,What is the boy doing in the beginning of the video?,v_51faGoLPJgM_0,The boy is chopping wood next to a pile of chopped logs while wearing blue jeans and a sweater.,0 415 | 413,v_AB480dHyDeM,What does the man do after removing the insole and laces from the shoe?,v_AB480dHyDeM_0,The man first removes the insole and laces from the shoe before putting some cleaning solution on the brush and dipping it in the water bowl to start brushing the shoe.,0 416 | 414,v_lPw_BRbPw5s,What is the attire of the man and woman dancing in the alley?,v_lPw_BRbPw5s_0,"The man is dressed in a blue T-shirt and a blue cap, while the woman is wearing a brown top and blue jeans.",0 417 | 415,v_Kofjs0rkwxc,What are the ingredients used by the chef in making sandwiches?,v_Kofjs0rkwxc_0,"The chef uses bread, butter, sliced turkey, cranberry sauce, and green leaves to make sandwiches. For the turkey sandwich, he spreads butter on the bread, places sliced turkey, adds cranberry sauce and green leaves and covers it with the other slice.",0 418 | 416,v_jiM0HUFn0e8,What are the little kids doing during the ballet practice?,v_jiM0HUFn0e8_0,"The little kids are sitting a few feet away from the trio of dancers, mimicking the hand moves and following the movements of the instructor and the other two dancers.",0 419 | 417,v_4oi6dXfpJS4,What color outfit is the gymnast wearing when she starts performing on the balance beam?,v_4oi6dXfpJS4_0,The gymnast is wearing a pink outfit when she mounts the balance beam.,0 420 | 418,v_4KMG6lEDSIM,What happens in the final scene of the video?,v_4KMG6lEDSIM_0,"In the final scene of the video, a man wearing a black gi breaks tiles placed on two cinder blocks.",0 421 | 419,v_889_kqLFM0E,What is the common theme among all the activities shown in the video?,v_889_kqLFM0E_0,All people in the video are engaged in different forms of rock climbing activities.,0 422 | 420,v_1YU4MSK80cQ,What exercises did the woman do in the video?,v_1YU4MSK80cQ_0,"The woman did exercises like swaying, moving back using her hands, doing the split, and swinging using the bars.",0 423 | 421,v_IDuoF51IMJ8,What is happening in the beginning of the video?,v_IDuoF51IMJ8_0,The video begins on a basketball court where two teams are playing dodgeball.,0 424 | 422,v_jkrlHzTRTvo,What activity are the young boys in the beginning of the video preparing for?,v_jkrlHzTRTvo_0,"The young boys are getting ready for a paintball game, and they are seen in a room putting on their gear and taking defensive positions in an arena with various inflatable bunkers used as obstacles.",0 425 | 423,v_dhsSq0K1xEg,What is written on the saxophone case?,v_dhsSq0K1xEg_0,'GUNS-N-ROSES' with a rose drawn underneath is written on the decorated saxophone case.,0 426 | 424,v_0x6ZV9Y8g_c,What is the toddler wearing in the beginning of the video?,v_0x6ZV9Y8g_c_0,"In the beginning of the video, the toddler is seen wearing blue and red pajamas.",0 427 | 425,v_-IMXSEIabMM,What are the people in the beginning of the video doing?,v_-IMXSEIabMM_0,The people in the beginning of the video are wearing winter clothes and shoveling snow around a car parked in front of a garage in the daytime.,0 428 | 426,v_dVpHF8tE2qo,What is the man wearing while lifting the barbell in the gym?,v_dVpHF8tE2qo_0,The man is wearing a red vest while lifting the barbell in the gym.,0 429 | 427,v_Kic33dtLja8,What type of dance are the women in the video performing?,v_Kic33dtLja8_0,"The women in the video are belly dancing. They move their hands, from side to side, forward and back while turning sideways.",0 430 | 428,v_4mRkWNLE66E,What is the number of the player being pointed to by the red arrow?,v_4mRkWNLE66E_0,The player in the white number 15 jersey is being pointed to by the red arrow and he scored a goal and celebrated with his teammates at the end of the video.,0 431 | 429,v_FOZdNPQ7d5Q,What is happening in the video?,v_FOZdNPQ7d5Q_0,"A man is getting a tattoo on his left arm from a woman in a dimly lit room. The woman is wearing a purple check shirt with black gloves and using a tattoo gun. The man is wearing a black beanie cap, a chain, and blue jeans. There are plants outside the window. The video ends with a close-up shot of the tattoo.",0 432 | 430,v_BSwChZlOb0k,What happens when the man tries to drink the flaming shot?,v_BSwChZlOb0k_0,"The flames from the shot glass stick to his mouth and he throws the glass on the table, which causes the table and a man standing near it to catch fire.",0 433 | 431,v_cqxiLraI8uY,What does the man wearing a red t-shirt do in the gym?,v_cqxiLraI8uY_0,The man lifts a barbell and holds it over his head before dropping it to the floor in a gym with black floors and fixed equipment.,0 434 | 432,v_d6PdD2O3Yrs,What is the boy doing in the video?,v_d6PdD2O3Yrs_0,The boy in black tights is practicing gymnastics by balancing and swinging on horizontal bars in the well-lit gymnasium.,0 435 | 433,v_3V9tzjyr51I,What is the man on the right holding in the video?,v_3V9tzjyr51I_0,The man on the right is holding a racquet in the video.,0 436 | 434,v_di5f5hM05Vw,Where is the man practicing Tai Chi?,v_di5f5hM05Vw_0,"The man is practicing Tai Chi movements in an indoor studio with well-lit light wood flooring, a mirror covering the wall at his back, and a window covered in purple drapes visible on the right side wall.",0 437 | 435,v_JN1wuOpCZvk,What are the men wearing in the video?,v_JN1wuOpCZvk_0,"The men are wearing black briefs with the one on the left wearing a grey sweatshirt and the one on the right, a white vest.",0 438 | 436,v_JH9qiuPCkY4,What is the color of the logo on the wall in the dirt bike yard at the beginning of the video?,v_JH9qiuPCkY4_0,There is a wall with a red logo on it in the dirt bike yard at the beginning of the video.,0 439 | 437,v_aQ-F9wr0HQ4,What is being sprayed in the video?,v_aQ-F9wr0HQ4_0,The 'Favor' spray bottle is being used in the video.,0 440 | 438,v_0UTrTg3DF5g,What drink is Pinky making in the video?,v_0UTrTg3DF5g_0,"Pinky is making a cocktail called 'GIN E HENDRIX' and she uses a variety of ingredients including cucumber, lime, lemon, basil, and different bottles of alcohol visible behind her on glass shelves.",0 441 | 439,v_Ie09KEAPP2I,What is the color of the labrador retriever in the video?,v_Ie09KEAPP2I_0,The labrador retriever in the video is brown.,0 442 | 440,v_Jdu5AqaWfz8,What colour is the bungee jump platform and where is the yellow inflatable boat located?,v_Jdu5AqaWfz8_0,"The bungee jump platform is yellow and the inflatable boat is located to the left near the shore. Three men, with two attendants in green shirts and one jumper wearing a t-shirt and a strap, are standing on the platform during the day. One of the attendants is belaying the bungee cord. After jumping off the platform and being pulled back up, the man is slowly reeled down and drawn to the shore by two attendants.",0 443 | 441,v_Erje-PwY9hE,What color are the kayaks tied to the pier and what is the color of the shirt worn by the person kayaking down the river?,v_Erje-PwY9hE_0,The kayaks at the pier are yellow and the person kayaking down the river is wearing a yellow shirt. ,0 444 | 442,v_HgYuRKZ_V90,What color uniform is one of the teams wearing in the video?,v_HgYuRKZ_V90_0,One of the teams is wearing a blue uniform and the other is wearing white.,0 445 | 443,v__jV5sAOOHLk,What is Brandt repairing in the video?,v__jV5sAOOHLk_0,"Brandt is repairing a punctured tire on a bicycle by unscrewing the derailleur guard, removing the tire, inserting a new tube, inflating it using a pump, fixing it back to the bicycle with a wrench and reapplying the rear brake.",0 446 | 444,v_JDwd6Ec454A,What color is the rabbit and what is he doing?,v_JDwd6Ec454A_0,The rabbit has a reddish coat with dark brown highlights and is grooming himself in a wooden box with a mesh floor. A person reaches in and pets the rabbit.,0 447 | 445,v_34SBoGK8mT8,What is the little girl doing in the video?,v_34SBoGK8mT8_0,"The little girl is playing hopscotch on a court drawn on the ground, picking up and throwing a pebble a few times before successfully landing it in a square and hopping to that square.",0 448 | 446,v_fNhw7xE3NM8,What is the girl wearing who is making the video?,v_fNhw7xE3NM8_0,"The girl making the video is wearing a black top and the room has printed walls with a chandelier, dolls, a green sofa, and a big green plant. There is also a white door present.",0 449 | 447,v_83f0iwb8VW8,What is the color of the carpet on the floor of the dojo?,v_83f0iwb8VW8_0,The floor of the dojo is made of blue and red carpet.,0 450 | 448,v_Lb7Q8qv3MHU,What does the girl do after taking a sip of mouthwash?,v_Lb7Q8qv3MHU_0,"The girl picks up her toothbrush, squeezes toothpaste on it, and starts brushing her teeth.",0 451 | 449,v_7dXqXFoju2M,Where does the video begin?,v_7dXqXFoju2M_0,The video begins in a dimly lit bathroom with a bathtub visible in the background. A woman stands in front of a mirror with a drinking glass in one hand and a cigarette in the other.,0 452 | 450,v_A20uQSBtYMs,What is the name written on the banner behind the men playing saxophones?,v_A20uQSBtYMs_0,The banner behind the men playing saxophones reads 'TWIN JAZZ'.,0 453 | 451,v_9InwA-ajB2M,Who is present inside the enclosure?,v_9InwA-ajB2M_0,A man wearing a cowboy hat and an orange shirt is standing in the middle of the enclosure and looking at the rider.,0 454 | 452,v_2nDbYhgdw2A,What type of dance moves are performed by the person wearing a yellow shirt in the beginning of the video?,v_2nDbYhgdw2A_0,The person in the yellow shirt performs the backspin as the first dance move.,0 455 | 453,v_eRco7yXc-40,What type of dive did the person perform off the diving board?,v_eRco7yXc-40_0,The person performed a reverse somersault dive off the diving board.,0 456 | 454,v_IGZvj1J9tNc,What drill are the players performing in the video?,v_IGZvj1J9tNc_0,"The players are running across the court in groups of three, while the one in the middle dribbles the ball and shoots it in the basket, and then they repeat the drill with a different player among the three, while the coach watches from outside the court.",0 457 | 455,v_49PbyLbrxlE,What is the woman doing in the bathroom?,v_49PbyLbrxlE_0,"The woman is washing a piece of cloth in front of the mirror and sink in a bathroom while wearing a blue top. Toiletries and a bottle of dish soap are on a shelf in front of the mirror and walls are lined with white tiles. As she continues to wash the cloth, she rinses it in the sink, which is filled with dirty water.",0 458 | 456,v_4QRRxub6L5Q,What is happening in the video with the girls and the foam platform?,v_4QRRxub6L5Q_0,The video shows a girl performing multiple backflips on a foam platform indoors while another girl watches. There is a poster of a cartoon sloth on the far wall. The video ends with the girl doing backflips and landing.,0 459 | 457,v_0DFz3sgfda0,What are the girls doing with the food items in the kitchen?,v_0DFz3sgfda0_0,"The girls are making a salad in the kitchen with various ingredients like lettuce, tomatoes, and peanuts.",0 460 | 458,v_cCDffwsJvsY,What does the person do to start the fire?,v_cCDffwsJvsY_0,The person lights a piece of chips and then uses it as kindling to light the arranged firewood. They also use a Bic lighter and light a few more pieces of kindling to help the fire light faster.,0 461 | 459,v_11p9goB2rm8,What activities were the teenagers engaged in?,v_11p9goB2rm8_0,The teenagers were hopping through the hopscotch court and attempting a pull-up from the basketball hoop before walking away from it. They were in a playground with multiple basketball hoops visible in the distance.,0 462 | 460,v_4Gx_8vuxzAk,What is the woman in the video cleaning the coffee table with?,v_4Gx_8vuxzAk_0,The woman is using wood polish and a towel to clean the coffee table.,0 463 | 461,v_HMs703DhjQU,What are the men wearing in the kitchen?,v_HMs703DhjQU_0,"The man on the left is wearing a black suit, and the man on the right is Will Smith, the actor wearing a multi-colored shirt.",0 464 | 462,v_Jy93pLzV2uw,Which team wins the exchange and celebrates?,v_Jy93pLzV2uw_0,The team wearing the orange uniform wins the exchange and celebrates.,0 465 | 463,v_6wPVO7I8iIY,What is the bartender doing in the video?,v_6wPVO7I8iIY_0,"The bartender is making a cocktail by picking up various bottles of alcohol, adding flavoring agent, ice cubes, and stirring the mixture in a mixing glass. He then empties the cocktail glass of ice and pours the drink into it. The video shows a cutaway of the ingredients for the drink. At the end of the video, the bartender is seen wiping the bar.",0 466 | 464,v_Fb4X1PWfOvY,What color clothes is the woman wearing while exercising on the parallel bars in the park?,v_Fb4X1PWfOvY_0,"The woman is wearing a neon green T-shirt, black track pants with white stripes, and white sports shoes.",0 467 | 465,v_IfugNwywj-k,What is the old man wearing while repairing the bicycle?,v_IfugNwywj-k_0,The old man is wearing a black vest over a white shirt while repairing the bicycle in a well-lit shop.,0 468 | 466,v_dZsk5Dp_5FE,What is the boy wearing while chopping wood?,v_dZsk5Dp_5FE_0,The boy is wearing an off-white T-shirt and shorts while chopping wood.,0 469 | 467,v_AL1W3HkjJ2Y,What is the color of the woman's outfit in the video?,v_AL1W3HkjJ2Y_0,The woman is wearing black trousers and a black top while holding a bow with a quiver full of arrows.,0 470 | 468,v_j_1kNiGcepU,"What is the man wearing who is playing the harmonica, and what is the man next to him doing?",v_j_1kNiGcepU_0,"The harmonica player is wearing a checkered shirt and has a backpack on. The man next to him in a hoodie is standing with his hands in his pockets, bobbing his head up and down before taking out his phone to record the harmonica player.",0 471 | 469,v_le7JdKQB65I,What color is the woman's coat and what color is her top?,v_le7JdKQB65I_0,The woman is wearing a white coat over a teal top.,0 472 | 470,v_-6dz6tBH77I,What is the man in the video holding in his hand?,v_-6dz6tBH77I_0,The man is holding a disk in his right hand.,0 473 | 471,v_BnO8AfQS8yE,What color is the shirt worn by the player who missed the shot?,v_BnO8AfQS8yE_0,The player who missed the shot is wearing a yellow T-shirt.,0 474 | 472,v_CvbBSwz-9XI,What is happening in the video?,v_CvbBSwz-9XI_0,A man is pouring water from a bucket on a dog standing under an open tap while shaking the water dry in slow motion.,0 475 | 473,v_-cAcA8dO7kA,What is happening in the beginning of the video?,v_-cAcA8dO7kA_0,A man is riding a dirt bike in a dirt field with houses visible beyond the field.,0 476 | 474,v_9snScUyE8_4,What is the woman doing in the video?,v_9snScUyE8_4_0,"The woman is washing dishes in the well-lit kitchen with a window behind her, while shelves of dishes are visible on one side and a house is visible through the window.",0 477 | 475,v_96ADEnJy4GU,What is happening in the scene with the monster truck?,v_96ADEnJy4GU_0,The monster truck is driving through a mud track and splashing mud as it turns.,0 478 | 476,v_-D1gdv_gQyw,What item does the man use to start the fire?,v_-D1gdv_gQyw_0,The man lights the stack of firewood on fire by striking a fire starter with his knife.,0 479 | 477,v__a527xEzhkw,What musical instrument is the girl holding?,v__a527xEzhkw_0,The girl is holding a harmonica.,0 480 | 478,v_cH3H-arDsH4,What is happening in the video?,v_cH3H-arDsH4_0,The video shows a game of water polo being played in a swimming pool with two teams trying to score goals.,0 481 | 479,v_hvhny4-sI-w,What are cameramen doing in the video?,v_hvhny4-sI-w_0,Cameramen with cameras on tripods are standing in front of the spectators capturing polo match going on in a polo field.,0 482 | 480,v_d09u-0k3LdE,What is the woman wearing and what is she holding?,v_d09u-0k3LdE_0,The woman is wearing a red vest and holding a hammer attached to a string. She is standing in a net in a field with a red mat on the ground. She swings the hammer with the string and releases it through the open end of the netting.,0 483 | 481,v_B0GCHDqEsRU,What is happening in the background while the couple is dancing?,v_B0GCHDqEsRU_0,"Children are running around in the background near the woods, and a man is tending to a barbeque a few feet behind the dancing couple. There is a red truck visible all the way back towards the fence.",0 484 | 482,v_aH39BSM0-Ag,What is the color of the jersey that Samuel Shearin is wearing?,v_aH39BSM0-Ag_0,Samuel Shearin is wearing a blue-green jersey with the number 24.,0 485 | 483,v_iqhdqhVttEM,What type of class is the man conducting?,v_iqhdqhVttEM_0,The man is leading a Zumba class where people are following his lead and dancing during the daytime and a crowd is gathered to watch.,0 486 | 484,v_cECycCDw1Nc,What are the ingredients shown in the video for the sandwich?,v_cECycCDw1Nc_0,"The ingredients for the sandwich in the video are a French roll, mayonnaise, pizza sauce, tomato slices, shredded lettuce, cheese, and pepperoni.",0 487 | 485,v_egowZOcbEkI,What is the instructor wearing?,v_egowZOcbEkI_0,The instructor is wearing a white shirt and green shorts.,0 488 | 486,v_HURRgbZGz8Q,What is happening in the outdoor stadium in the video?,v_HURRgbZGz8Q_0,"The video shows a spear-throw competition where athletes holding spears run and throw them in an outdoor stadium filled with spectators. The landing area has numbered lanes, and a man in a suit is seen near the number 70 marker. The scoreboard with names and throw distances is shown towards the end.",0 489 | 487,v_afMQA352X_A,What is the trainer wearing during the kickboxing practice?,v_afMQA352X_A_0,The trainer is wearing focus mitts on both hands during the kickboxing practice.,0 490 | 488,v_l8gg9TYh_B0,What are the news anchors wearing?,v_l8gg9TYh_B0_0,The woman anchor is wearing a black top and the man anchor is wearing a black suit.,0 491 | 489,v_Ds2O0-8Labs,What is the weight of the heaviest barbell the man lifts?,v_Ds2O0-8Labs_0,The man lifts a barbell of 130 kilograms at the end.,0 492 | 490,v_k1QAWLupB-I,What branded item are the people holding in the beginning of the video?,v_k1QAWLupB-I_0,"The people at the beginning are holding an Old Spice-branded ping pong paddle, and the woman is holding a paddle in front of her face with a picture of the man she is standing next to on it.",0 493 | 491,v_9UvVeUYQLgA,What are some of the structures found in the paintball arena?,v_9UvVeUYQLgA_0,"Cable drums, old tires, weeds, and cinderblocks are used as cover in the paintball arena.",0 494 | 492,v_dEpUIpmBvow,What time of the day is the man climbing the rock?,v_dEpUIpmBvow_0,The man is climbing the rock during daytime.,0 495 | 493,v_EZyLBq2jJtE,What is Bethany doing when the video starts?,v_EZyLBq2jJtE_0,"Bethany is shaving her legs while wearing a red T-shirt, black & purple short track pants, white sports shoes, rings on both hands, and a bracelet on her left hand. She is also speaking to the cameraman.",0 496 | 494,v_2fzIi1lPd8U,Who is playing with the two kids in the video?,v_2fzIi1lPd8U_0,The video shows an elderly woman playing with a little boy wearing a red shirt and a little girl standing on the side of the porch.,0 497 | 495,v_C7yd6yEkxXE,What are some visible features on the dirt bike track?,v_C7yd6yEkxXE_0,The video shows a dirt bike track surrounded by trees and vegetation with a cloudy sky in the background.,0 498 | 496,v_IN23CO8Kotk,What is the parrot eating from the pink coffee cup on the kitchen counter?,v_IN23CO8Kotk_0,The parrot is eating cream from a pink coffee cup with a butterfly painted on it.,0 499 | 497,v_BFChXuwZ4wo,What did the woman get tattooed on her wrist?,v_BFChXuwZ4wo_0,The woman got a stencil tattooed on her wrist at the tattoo parlor.,0 500 | 498,v_9k19UUJtldg,What is happening in the first part of the video?,v_9k19UUJtldg_0,"In the first scene, the boy is playing ping-pong alone in a room.",0 501 | -------------------------------------------------------------------------------- /eval_gpt4v_multiplechoice.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (c) 2024, Deep Representation Learning Research Group, Seoul National University. 3 | All rights reserved. 4 | SPDX-License-Identifier: BSD-3-Clause 5 | For full license text, see the LICENSE file in the repo root or https://opensource.org/licenses/BSD-3-Clause 6 | """ 7 | 8 | import sys, os 9 | import time 10 | from io import BytesIO 11 | import argparse 12 | import re 13 | import uuid 14 | 15 | sys.path.append(os.path.dirname(os.path.abspath(os.path.dirname(__file__)))) 16 | 17 | from vision_processor.fps_gridview_processor import * 18 | 19 | from pipeline_processor.gpt4_pipeline import * 20 | from evaluation.direct_answer_eval import * 21 | 22 | 23 | def infer_and_eval_model(args): 24 | path_qa = args.path_qa_pair_csv 25 | path_video = args.path_video 26 | path_result_dir = args.path_result 27 | api_key = args.api_key 28 | 29 | system_prompt, user_prompt, assistant_prompt = get_prompt() 30 | frame_fixed_number = 6 31 | 32 | # In case of NExT-QA, TVQA, IntentQA and EgoSchema, These has five options on multiple-choice 33 | func_user_prompt = lambda prompt, row: prompt % ( 34 | row["question"], 35 | row["a0"], 36 | row["a1"], 37 | row["a2"], 38 | row["a3"], 39 | row["a4"], 40 | ) 41 | 42 | # In case of STAR benchamrk, with four options on multiple-choice, please use the following codes and select prompt according to llm size. 43 | """ 44 | func_user_prompt = lambda prompt, row: prompt % ( 45 | row["question"], 46 | row["a0"], 47 | row["a1"], 48 | row["a2"], 49 | row["a3"], 50 | ) 51 | user_prompt = "The provided image arranges key frames from a video in a grid view. They are arranged in chronological order, holding temporal information from the top left to the bottom right. You need to choose one of the following five options to answer the question, '%s?' : A.'%s', B.'%s', C.'%s', D.'%s'. Please provide a single-character answer (A, B, C, or D) to the multiple-choice question, and your answer must be one of the letters (A, B, C or D). Your response must only contain one character without any other string." 52 | """ 53 | 54 | print("loading model") 55 | 56 | gpt4vPipeline = Gpt4Pipeline( 57 | path_qa, 58 | path_video, 59 | dir=path_result_dir, 60 | ) 61 | gpt4vPipeline.set_component( 62 | api_key, 63 | system_prompt, 64 | user_prompt, 65 | assistant_prompt, 66 | frame_fixed_number=frame_fixed_number, 67 | func_user_prompt=func_user_prompt, 68 | ) 69 | df_merged, path_df_merged = gpt4vPipeline.do_pipeline() 70 | 71 | print("gpt4 prediction result : " + path_df_merged) 72 | print("start multiple-choice evaluation") 73 | 74 | eval_multiple_choice_gpt4(df_merged) 75 | 76 | 77 | def get_prompt(): 78 | system_prompt = "You are an intelligent chatbot designed for understanding videos with key frames in grid view.\n-----\n##INSTRUCTIONS:\n- Focus and answer the question on the meaningful spatial and temporal information in grid view refering to question \n- Even if you're not sure of the answer, please choose the most likely option and answer it" 79 | user_prompt = "The provided image arranges key frames from a video in a grid view. They are arranged in chronological order, holding temporal information from the top left to the bottom right. You need to choose one of the following five options to answer the question, '%s?' : 0.'%s', 1.'%s', 2.'%s', 3.'%s', 4.'%s'. Please provide a single-number answer (0, 1, 2, 3 or 4) to the multiple-choice question, and your answer must be one of the letters (0, 1, 2, 3 or 4). Your response must only contain one number without any string." 80 | assistant_prompt = "" 81 | return system_prompt, user_prompt, assistant_prompt 82 | 83 | 84 | def validate_video_path(filename): 85 | pattern = r"\.(avi|mp4|mkv|gif|webm)$" # %s.avi 또는 %s.mp4 형식을 따르는지 확인하는 정규 표현식 86 | if not re.search(pattern, filename): 87 | raise argparse.ArgumentTypeError( 88 | f"No valid video path. You must include %s and the extension of video file. (e.g., /tmp/%s.mp4)" 89 | ) 90 | return filename 91 | 92 | 93 | if __name__ == "__main__": 94 | parser = argparse.ArgumentParser(description="LLaVA v1.6 with IG-VLM") 95 | parser.add_argument( 96 | "--path_qa_pair_csv", 97 | type=str, 98 | required=True, 99 | help="path of question and answer. It should be csv files", 100 | ) 101 | parser.add_argument( 102 | "--path_video", 103 | type=validate_video_path, 104 | required=True, 105 | metavar="/tmp/%s.mp4", 106 | help="path of video files. You must include string format specifier and the extension of video file.", 107 | ) 108 | parser.add_argument( 109 | "--path_result", type=str, required=True, help="path of output directory" 110 | ) 111 | 112 | parser.add_argument( 113 | "--api_key", 114 | type=str, 115 | required=True, 116 | help="api key for gpt-4v", 117 | ) 118 | 119 | args = parser.parse_args() 120 | 121 | infer_and_eval_model(args) 122 | -------------------------------------------------------------------------------- /eval_gpt4v_openended.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (c) 2024, Deep Representation Learning Research Group, Seoul National University. 3 | All rights reserved. 4 | SPDX-License-Identifier: BSD-3-Clause 5 | For full license text, see the LICENSE file in the repo root or https://opensource.org/licenses/BSD-3-Clause 6 | """ 7 | 8 | import sys, os 9 | import time 10 | from io import BytesIO 11 | import argparse 12 | import re 13 | import uuid 14 | 15 | sys.path.append(os.path.dirname(os.path.abspath(os.path.dirname(__file__)))) 16 | 17 | from vision_processor.fps_gridview_processor import * 18 | from pipeline_processor.gpt4_pipeline import * 19 | from evaluation.gpt3_evaluation_utils import * 20 | 21 | 22 | def infer_and_eval_model(args): 23 | path_qa = args.path_qa_pair_csv 24 | path_video = args.path_video 25 | path_result_dir = args.path_result 26 | api_key = args.api_key 27 | 28 | system_prompt, user_prompt, assistant_prompt = get_prompt() 29 | frame_fixed_number = 6 30 | 31 | print("loading model") 32 | 33 | gpt4vPipeline = Gpt4Pipeline( 34 | path_qa, 35 | path_video, 36 | dir=path_result_dir, 37 | ) 38 | gpt4vPipeline.set_component( 39 | api_key, 40 | system_prompt, 41 | user_prompt, 42 | assistant_prompt, 43 | frame_fixed_number=frame_fixed_number, 44 | ) 45 | df_merged, path_df_merged = gpt4vPipeline.do_pipeline() 46 | 47 | print("GPT-4V prediction result : " + path_df_merged) 48 | print("start gpt3-evaluation") 49 | 50 | gpt3_dir = os.path.join(path_result_dir, "results_gpt3_evaluation") 51 | 52 | df_qa, path_merged = eval_gpt3(df_merged, gpt3_dir, api_key) 53 | 54 | print("Gpt-3-evaluation file : " + path_merged) 55 | yes_count = df_qa[df_qa["gpt3_pred"] == "yes"].shape[0] 56 | score = df_qa["gpt3_score"].mean() 57 | 58 | print("Acc : %s" % (str(yes_count / df_qa.shape[0]))) 59 | print("Score : %s" % (str(score))) 60 | 61 | 62 | def get_prompt(): 63 | system_prompt = "" 64 | user_prompt = "The provided image arranges keyframes from a video in a grid view. Answer concisely with overall content and context of the video, highlighting any significant events, characters, or objects that appear throughout the frames. Question: %s?" 65 | assistant_prompt = "In the video," 66 | return system_prompt, user_prompt, assistant_prompt 67 | 68 | 69 | def validate_video_path(filename): 70 | pattern = r"\.(avi|mp4|mkv|gif|webm)$" # %s.avi 또는 %s.mp4 형식을 따르는지 확인하는 정규 표현식 71 | if not re.search(pattern, filename): 72 | raise argparse.ArgumentTypeError( 73 | f"No valid video path. You must include %s and the extension of video file. (e.g., /tmp/%s.mp4)" 74 | ) 75 | return filename 76 | 77 | 78 | if __name__ == "__main__": 79 | parser = argparse.ArgumentParser(description="LLaVA v1.6 with IG-VLM") 80 | parser.add_argument( 81 | "--path_qa_pair_csv", 82 | type=str, 83 | required=True, 84 | help="path of question and answer. It should be csv files", 85 | ) 86 | parser.add_argument( 87 | "--path_video", 88 | type=validate_video_path, 89 | required=True, 90 | metavar="/tmp/%s.mp4", 91 | help="path of video files. You must include string format specifier and the extension of video file.", 92 | ) 93 | parser.add_argument( 94 | "--path_result", type=str, required=True, help="path of output directory" 95 | ) 96 | 97 | parser.add_argument( 98 | "--api_key", 99 | type=str, 100 | required=True, 101 | help="api key for gpt-4v and gpt-3 evaluation", 102 | ) 103 | args = parser.parse_args() 104 | 105 | infer_and_eval_model(args) 106 | -------------------------------------------------------------------------------- /eval_gpt4v_textgeneration_openended.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (c) 2024, Deep Representation Learning Research Group, Seoul National University. 3 | All rights reserved. 4 | SPDX-License-Identifier: BSD-3-Clause 5 | For full license text, see the LICENSE file in the repo root or https://opensource.org/licenses/BSD-3-Clause 6 | """ 7 | 8 | import sys, os 9 | import time 10 | from io import BytesIO 11 | import argparse 12 | import re 13 | import uuid 14 | 15 | sys.path.append(os.path.dirname(os.path.abspath(os.path.dirname(__file__)))) 16 | 17 | from vision_processor.fps_gridview_processor import * 18 | 19 | from pipeline_processor.gpt4_pipeline import * 20 | from evaluation.gpt3_evaluation_utils import * 21 | 22 | 23 | def infer_and_eval_model(args): 24 | path_qa = args.path_qa_pair_csv 25 | path_video = args.path_video 26 | path_result_dir = args.path_result 27 | api_key = args.api_key 28 | 29 | system_prompt, user_prompt, assistant_prompt = get_prompt() 30 | frame_fixed_number = 6 31 | 32 | print("loading model") 33 | 34 | gpt4vPipeline = Gpt4Pipeline( 35 | path_qa, 36 | path_video, 37 | dir=path_result_dir, 38 | ) 39 | gpt4vPipeline.set_component( 40 | api_key, 41 | system_prompt, 42 | user_prompt, 43 | assistant_prompt, 44 | frame_fixed_number=frame_fixed_number, 45 | ) 46 | df_merged, path_df_merged = gpt4vPipeline.do_pipeline() 47 | 48 | print("GPT-4V prediction result : " + path_df_merged) 49 | print("start gpt3-evaluation") 50 | 51 | gpt3_dir = os.path.join(path_result_dir, "results_gpt3_evaluation") 52 | 53 | # Correctness Information(CI) Evaluation on Text Genearation Performance 54 | df_qa, path_merged = eval_gpt3( 55 | df_merged, gpt3_dir, api_key, gpt_eval_type=EvaluationType.CORRECTNESS 56 | ) 57 | 58 | """ 59 | Text Generation Benchmark has five evaluations. CI(Correctness Information), DO(Detailed Orientation), CU(Context Understanding), TU(Temporal Understanding), CO(Consistency) 60 | 61 | In case of Do, Please use the following. 62 | # df_qa, path_merged = eval_gpt3(df_merged, gpt3_dir, api_key, gpt_eval_type=EvaluationType.DETAILED_ORIENTATION) 63 | 64 | In case of CU, Please use the following. 65 | # df_qa, path_merged = eval_gpt3(df_merged, gpt3_dir, api_key, gpt_eval_type=EvaluationType.CONTEXT) 66 | 67 | In case of TU, Please use the following. 68 | # df_qa, path_merged = eval_gpt3(df_merged, gpt3_dir, api_key, gpt_eval_type=EvaluationType.TEMPORAL) 69 | 70 | In case of CO, we need to evaluate llava twice on two questions. Then, Please use the following 71 | # df_qa, path_merged = eval_gpt3_consistency(df_merged1, df_merged2, gpt3_dir, api_key) 72 | 73 | """ 74 | 75 | print("Gpt-3-evaluation file : " + path_merged) 76 | yes_count = df_qa[df_qa["gpt3_pred"] == "yes"].shape[0] 77 | score = df_qa["gpt3_score"].mean() 78 | 79 | print("Acc : %s" % (str(yes_count / df_qa.shape[0]))) 80 | print("Score : %s" % (str(score))) 81 | 82 | 83 | def get_prompt(): 84 | system_prompt = "" 85 | user_prompt = "The provided image arranges keyframes from a video in a grid view. Answer concisely with overall content and context of the video, highlighting any significant events, characters, or objects that appear throughout the frames. Question: %s?" 86 | assistant_prompt = "In the video," 87 | return system_prompt, user_prompt, assistant_prompt 88 | 89 | 90 | def validate_video_path(filename): 91 | pattern = r"\.(avi|mp4|mkv|gif|webm)$" # %s.avi 또는 %s.mp4 형식을 따르는지 확인하는 정규 표현식 92 | if not re.search(pattern, filename): 93 | raise argparse.ArgumentTypeError( 94 | f"No valid video path. You must include %s and the extension of video file. (e.g., /tmp/%s.mp4)" 95 | ) 96 | return filename 97 | 98 | 99 | if __name__ == "__main__": 100 | parser = argparse.ArgumentParser(description="LLaVA v1.6 with IG-VLM") 101 | parser.add_argument( 102 | "--path_qa_pair_csv", 103 | type=str, 104 | required=True, 105 | help="path of question and answer. It should be csv files", 106 | ) 107 | parser.add_argument( 108 | "--path_video", 109 | type=validate_video_path, 110 | required=True, 111 | metavar="/tmp/%s.mp4", 112 | help="path of video files. You must include string format specifier and the extension of video file.", 113 | ) 114 | parser.add_argument( 115 | "--path_result", type=str, required=True, help="path of output directory" 116 | ) 117 | 118 | parser.add_argument( 119 | "--api_key", 120 | type=str, 121 | required=True, 122 | help="api key for gpt-4v and gpt-3 evaluation", 123 | ) 124 | args = parser.parse_args() 125 | 126 | infer_and_eval_model(args) 127 | -------------------------------------------------------------------------------- /eval_llava_multiplechoice.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (c) 2024, Deep Representation Learning Research Group, Seoul National University. 3 | All rights reserved. 4 | SPDX-License-Identifier: BSD-3-Clause 5 | For full license text, see the LICENSE file in the repo root or https://opensource.org/licenses/BSD-3-Clause 6 | """ 7 | 8 | import sys, os 9 | import time 10 | from io import BytesIO 11 | import argparse 12 | import re 13 | import uuid 14 | 15 | sys.path.append(os.path.dirname(os.path.abspath(os.path.dirname(__file__)))) 16 | 17 | from vision_processor.fps_gridview_processor import * 18 | 19 | from pipeline_processor.llava_pipeline import * 20 | from evaluation.direct_answer_eval import * 21 | 22 | 23 | def infer_and_eval_model(args): 24 | path_qa = args.path_qa_pair_csv 25 | path_video = args.path_video 26 | path_result_dir = args.path_result 27 | llm_size = args.llm_size 28 | 29 | model_name, user_prompt = get_llava_and_prompt(llm_size) 30 | frame_fixed_number = 6 31 | 32 | # In case of NExT-QA, TVQA, IntentQA and EgoSchema, user the following codes. 33 | func_user_prompt = lambda prompt, row: prompt % ( 34 | row["question"], 35 | row["a0"], 36 | row["a1"], 37 | row["a2"], 38 | row["a3"], 39 | row["a4"], 40 | ) 41 | 42 | # In case of STAR benchamrk, use the following codes and select prompt according to llm size. 43 | """ 44 | func_user_prompt = lambda prompt, row: prompt % ( 45 | row["question"], 46 | row["a0"], 47 | row["a1"], 48 | row["a2"], 49 | row["a3"] 50 | ) 51 | # 7b 52 | user_prompt = "Select correct option to answer the question. USER: \nThe provided image arranges keyframes from a video in a grid view. Question: %s A:%s. B:%s. C:%s. D:%s. \nSelect the correct answer from the options. \nASSISTANT:\nAnswer:" 53 | # 13b 54 | user_prompt = "USER: \nThe provided image arranges keyframes from a video in a grid view. Question: %s?\n A:%s. B:%s. C:%s. D:%s. \n Select the correct answer from the options(A,B,C,D). \nASSISTANT: \nAnswer:" 55 | # 34b 56 | user_prompt = "<|im_start|>system\n Select correct option to answer the question.<|im_end|>\n<|im_start|>user\n \n Question: %s? A:%s. B:%s. C:%s. D:%s. Select the correct answer from the options. <|im_end|>\n<|im_start|>assistant\nAnswer:" 57 | """ 58 | 59 | print("loading [%s]" % (model_name)) 60 | 61 | llavaPipeline = LlavaPipeline( 62 | model_name, 63 | path_qa, 64 | path_video, 65 | dir=path_result_dir, 66 | ) 67 | llavaPipeline.set_component( 68 | user_prompt, 69 | frame_fixed_number=frame_fixed_number, 70 | func_user_prompt=func_user_prompt, 71 | ) 72 | df_merged, path_df_merged = llavaPipeline.do_pipeline() 73 | 74 | print("llava prediction result : " + path_df_merged) 75 | print("start multiple-choice evaluation") 76 | 77 | eval_multiple_choice(df_merged) 78 | 79 | 80 | def get_llava_and_prompt(llm_size): 81 | if llm_size in ["7b"]: 82 | prompt = "Select correct option to answer the question. USER: \nThe provided image arranges keyframes from a video in a grid view. Question: %s A:%s. B:%s. C:%s. D:%s. E:%s. \nSelect the correct answer from the options. \nASSISTANT:\nAnswer:" 83 | model_name = "llava-v1.6-vicuna-%s" % (llm_size) 84 | elif llm_size in ["13b"]: 85 | prompt = "USER: \nThe provided image arranges keyframes from a video in a grid view. Question: %s?\n A:%s. B:%s. C:%s. D:%s. E:%s. \n Select the correct answer from the options(A,B,C,D,E). \nASSISTANT: \nAnswer:" 86 | model_name = "llava-v1.6-vicuna-%s" % (llm_size) 87 | else: 88 | prompt = "<|im_start|>system\n Select correct option to answer the question.<|im_end|>\n<|im_start|>user\n \n Question: %s? A:%s. B:%s. C:%s. D:%s. E: %s. Select the correct answer from the options. <|im_end|>\n<|im_start|>assistant\nAnswer:" 89 | model_name = "llava-v1.6-%s" % (llm_size) 90 | return model_name, prompt 91 | 92 | 93 | def validate_llm_size(type_llm_size): 94 | if type_llm_size not in {"7b", "13b", "34b"}: 95 | raise argparse.ArgumentTypeError(f"No valid LLM size.") 96 | return type_llm_size 97 | 98 | 99 | def validate_video_path(filename): 100 | pattern = r"\.(avi|mp4|mkv|gif|webm)$" # %s.avi 또는 %s.mp4 형식을 따르는지 확인하는 정규 표현식 101 | if not re.search(pattern, filename): 102 | raise argparse.ArgumentTypeError( 103 | f"No valid video path. You must include %s and the extension of video file. (e.g., /tmp/%s.mp4)" 104 | ) 105 | return filename 106 | 107 | 108 | if __name__ == "__main__": 109 | parser = argparse.ArgumentParser(description="LLaVA v1.6 with IG-VLM") 110 | parser.add_argument( 111 | "--path_qa_pair_csv", 112 | type=str, 113 | required=True, 114 | help="path of question and answer. It should be csv files", 115 | ) 116 | parser.add_argument( 117 | "--path_video", 118 | type=validate_video_path, 119 | required=True, 120 | metavar="/tmp/%s.mp4", 121 | help="path of video files. You must include string format specifier and the extension of video file.", 122 | ) 123 | parser.add_argument( 124 | "--path_result", type=str, required=True, help="path of output directory" 125 | ) 126 | 127 | parser.add_argument( 128 | "--llm_size", 129 | type=validate_llm_size, 130 | default="7b", 131 | help="You can choose llm size of LLaVA. 7b | 13b | 34b", 132 | ) 133 | 134 | args = parser.parse_args() 135 | 136 | infer_and_eval_model(args) 137 | -------------------------------------------------------------------------------- /eval_llava_openended.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (c) 2024, Deep Representation Learning Research Group, Seoul National University. 3 | All rights reserved. 4 | SPDX-License-Identifier: BSD-3-Clause 5 | For full license text, see the LICENSE file in the repo root or https://opensource.org/licenses/BSD-3-Clause 6 | """ 7 | 8 | import sys, os 9 | import time 10 | from io import BytesIO 11 | import argparse 12 | import re 13 | import uuid 14 | 15 | sys.path.append(os.path.dirname(os.path.abspath(os.path.dirname(__file__)))) 16 | 17 | from vision_processor.fps_gridview_processor import * 18 | 19 | from pipeline_processor.llava_pipeline import * 20 | from evaluation.gpt3_evaluation_utils import * 21 | 22 | 23 | def infer_and_eval_model(args): 24 | path_qa = args.path_qa_pair_csv 25 | path_video = args.path_video 26 | path_result_dir = args.path_result 27 | llm_size = args.llm_size 28 | api_key = args.api_key 29 | 30 | model_name, user_prompt = get_llava_and_prompt(llm_size) 31 | frame_fixed_number = 6 32 | 33 | print("loading [%s]" % (model_name)) 34 | 35 | llavaPipeline = LlavaPipeline( 36 | model_name, 37 | path_qa, 38 | path_video, 39 | dir=path_result_dir, 40 | ) 41 | llavaPipeline.set_component( 42 | user_prompt, 43 | frame_fixed_number=frame_fixed_number, 44 | ) 45 | df_merged, path_df_merged = llavaPipeline.do_pipeline() 46 | 47 | print("llava prediction result : " + path_df_merged) 48 | print("start gpt3-evaluation") 49 | 50 | gpt3_dir = os.path.join(path_result_dir, "results_gpt3_evaluation") 51 | 52 | df_qa, path_merged = eval_gpt3(df_merged, gpt3_dir, api_key) 53 | 54 | print("Gpt-3-evaluation file : " + path_merged) 55 | yes_count = df_qa[df_qa["gpt3_pred"] == "yes"].shape[0] 56 | score = df_qa["gpt3_score"].mean() 57 | 58 | print("Acc : %s" % (str(yes_count / df_qa.shape[0]))) 59 | print("Score : %s" % (str(score))) 60 | 61 | 62 | def get_llava_and_prompt(llm_size): 63 | if llm_size in ["7b", "13b"]: 64 | prompt = "USER: \nThe provided image arranges keyframes from a video in a grid view. Answer concisely with overall content and context of the video, highlighting any significant events, characters, or objects that appear throughout the frames. Question: %s? \nASSISTANT:\nAnswer: In the video," 65 | model_name = "llava-v1.6-vicuna-%s" % (llm_size) 66 | else: 67 | prompt = "<|im_start|>system\n Answer the question. <|im_end|>\n<|im_start|>user\n \nThe provided image arranges keyframes from a video in a grid view. Answer concisely with overall content and context of the video, highlighting any significant events, characters, or objects that appear throughout the frames. Question: %s? <|im_end|>\n<|im_start|>assistant\nAnswer: In the video," 68 | model_name = "llava-v1.6-%s" % (llm_size) 69 | return model_name, prompt 70 | 71 | 72 | def validate_llm_size(type_llm_size): 73 | if type_llm_size not in {"7b", "13b", "34b"}: 74 | raise argparse.ArgumentTypeError(f"No valid LLM size.") 75 | return type_llm_size 76 | 77 | 78 | def validate_video_path(filename): 79 | pattern = r"\.(avi|mp4|mkv|gif|webm)$" # %s.avi 또는 %s.mp4 형식을 따르는지 확인하는 정규 표현식 80 | if not re.search(pattern, filename): 81 | raise argparse.ArgumentTypeError( 82 | f"No valid video path. You must include %s and the extension of video file. (e.g., /tmp/%s.mp4)" 83 | ) 84 | return filename 85 | 86 | 87 | if __name__ == "__main__": 88 | parser = argparse.ArgumentParser(description="LLaVA v1.6 with IG-VLM") 89 | parser.add_argument( 90 | "--path_qa_pair_csv", 91 | type=str, 92 | required=True, 93 | help="path of question and answer. It should be csv files", 94 | ) 95 | parser.add_argument( 96 | "--path_video", 97 | type=validate_video_path, 98 | required=True, 99 | metavar="/tmp/%s.mp4", 100 | help="path of video files. You must include string format specifier and the extension of video file.", 101 | ) 102 | parser.add_argument( 103 | "--path_result", type=str, required=True, help="path of output directory" 104 | ) 105 | 106 | parser.add_argument( 107 | "--llm_size", 108 | type=validate_llm_size, 109 | default="7b", 110 | help="You can choose llm size of LLaVA. 7b | 13b | 34b", 111 | ) 112 | 113 | parser.add_argument( 114 | "--api_key", 115 | type=str, 116 | required=True, 117 | help="api key for gpt-3 evaluation", 118 | ) 119 | args = parser.parse_args() 120 | 121 | infer_and_eval_model(args) 122 | -------------------------------------------------------------------------------- /eval_llava_textgeneration_openended.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (c) 2024, Deep Representation Learning Research Group, Seoul National University. 3 | All rights reserved. 4 | SPDX-License-Identifier: BSD-3-Clause 5 | For full license text, see the LICENSE file in the repo root or https://opensource.org/licenses/BSD-3-Clause 6 | """ 7 | 8 | import sys, os 9 | import time 10 | from io import BytesIO 11 | import argparse 12 | import re 13 | import uuid 14 | 15 | sys.path.append(os.path.dirname(os.path.abspath(os.path.dirname(__file__)))) 16 | 17 | from vision_processor.fps_gridview_processor import * 18 | 19 | from pipeline_processor.llava_pipeline import * 20 | from evaluation.gpt3_evaluation_utils import * 21 | 22 | 23 | def infer_and_eval_model(args): 24 | path_qa = args.path_qa_pair_csv 25 | path_video = args.path_video 26 | path_result_dir = args.path_result 27 | llm_size = args.llm_size 28 | api_key = args.api_key 29 | 30 | model_name, user_prompt = get_llava_and_prompt(llm_size) 31 | frame_fixed_number = 6 32 | 33 | print("loading [%s]" % (model_name)) 34 | 35 | llavaPipeline = LlavaPipeline( 36 | model_name, 37 | path_qa, 38 | path_video, 39 | dir=path_result_dir, 40 | ) 41 | llavaPipeline.set_component( 42 | user_prompt, 43 | frame_fixed_number=frame_fixed_number, 44 | ) 45 | df_merged, path_df_merged = llavaPipeline.do_pipeline() 46 | 47 | print("llava prediction result : " + path_df_merged) 48 | print("start gpt3-evaluation") 49 | 50 | gpt3_dir = os.path.join(path_result_dir, "results_gpt3_evaluation") 51 | 52 | # Correctness Information Evaluation for Text Genearation Performance 53 | df_qa, path_merged = eval_gpt3( 54 | df_merged, gpt3_dir, api_key, gpt_eval_type=EvaluationType.CORRECTNESS 55 | ) 56 | 57 | """ 58 | Text Generation Benchmark has five evaluations. CI(Correctness Information), DO(Detailed Orientation), CU(Context Understanding), TU(Temporal Understanding), CO(Consistency) 59 | 60 | In case of Do, Please use the following. 61 | # df_qa, path_merged = eval_gpt3(df_merged, gpt3_dir, api_key, gpt_eval_type=EvaluationType.DETAILED_ORIENTATION) 62 | 63 | In case of CU, Please use the following. 64 | # df_qa, path_merged = eval_gpt3(df_merged, gpt3_dir, api_key, gpt_eval_type=EvaluationType.CONTEXT) 65 | 66 | In case of TU, Please use the following. 67 | # df_qa, path_merged = eval_gpt3(df_merged, gpt3_dir, api_key, gpt_eval_type=EvaluationType.TEMPORAL) 68 | 69 | In case of CO, we need to evaluate llava twice on two questions. Then, Please use the following 70 | # df_qa, path_merged = eval_gpt3_consistency(df_merged1, df_merged2, gpt3_dir, api_key) 71 | 72 | """ 73 | 74 | print("Gpt-3-evaluation file : " + path_merged) 75 | yes_count = df_qa[df_qa["gpt3_pred"] == "yes"].shape[0] 76 | score = df_qa["gpt3_score"].mean() 77 | 78 | print("Acc : %s" % (str(yes_count / df_qa.shape[0]))) 79 | print("Score : %s" % (str(score))) 80 | 81 | 82 | def get_llava_and_prompt(llm_size): 83 | if llm_size in ["7b", "13b"]: 84 | prompt = "USER: \nThe provided image arranges keyframes from a video in a grid view. Answer concisely with overall content and context of the video, highlighting any significant events, characters, or objects that appear throughout the frames. Question: %s? \nASSISTANT:\nAnswer: In the video," 85 | model_name = "llava-v1.6-vicuna-%s" % (llm_size) 86 | else: 87 | prompt = "<|im_start|>system\n Answer the question. <|im_end|>\n<|im_start|>user\n \nThe provided image arranges keyframes from a video in a grid view. Answer concisely with overall content and context of the video, highlighting any significant events, characters, or objects that appear throughout the frames. Question: %s? <|im_end|>\n<|im_start|>assistant\nAnswer: In the video," 88 | model_name = "llava-v1.6-%s" % (llm_size) 89 | return model_name, prompt 90 | 91 | 92 | def validate_llm_size(type_llm_size): 93 | if type_llm_size not in {"7b", "13b", "34b"}: 94 | raise argparse.ArgumentTypeError(f"No valid LLM size.") 95 | return type_llm_size 96 | 97 | 98 | def validate_video_path(filename): 99 | pattern = r"\.(avi|mp4|mkv|gif|webm)$" # %s.avi 또는 %s.mp4 형식을 따르는지 확인하는 정규 표현식 100 | if not re.search(pattern, filename): 101 | raise argparse.ArgumentTypeError( 102 | f"No valid video path. You must include %s and the extension of video file. (e.g., /tmp/%s.mp4)" 103 | ) 104 | return filename 105 | 106 | 107 | if __name__ == "__main__": 108 | parser = argparse.ArgumentParser(description="LLaVA v1.6 with IG-VLM") 109 | parser.add_argument( 110 | "--path_qa_pair_csv", 111 | type=str, 112 | required=True, 113 | help="path of question and answer. It should be csv files", 114 | ) 115 | parser.add_argument( 116 | "--path_video", 117 | type=validate_video_path, 118 | required=True, 119 | metavar="/tmp/%s.mp4", 120 | help="path of video files. You must include string format specifier and the extension of video file.", 121 | ) 122 | parser.add_argument( 123 | "--path_result", type=str, required=True, help="path of output directory" 124 | ) 125 | 126 | parser.add_argument( 127 | "--llm_size", 128 | type=validate_llm_size, 129 | default="7b", 130 | help="You can choose llm size of LLaVA. 7b | 13b | 34b", 131 | ) 132 | 133 | parser.add_argument( 134 | "--api_key", 135 | type=str, 136 | required=True, 137 | help="api key for gpt-3 evaluation", 138 | ) 139 | args = parser.parse_args() 140 | 141 | infer_and_eval_model(args) 142 | -------------------------------------------------------------------------------- /evaluation/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/evaluation/__init__.py -------------------------------------------------------------------------------- /evaluation/direct_answer_eval.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (c) 2024, Deep Representation Learning Research Group, Seoul National University. 3 | All rights reserved. 4 | SPDX-License-Identifier: BSD-3-Clause 5 | For full license text, see the LICENSE file in the repo root or https://opensource.org/licenses/BSD-3-Clause 6 | """ 7 | 8 | import pandas as pd 9 | 10 | 11 | def eval_multiple_choice(df): 12 | df["predicted_answer"] = df.apply(map_prediction_to_answer_v2, axis=1) 13 | df["is_correct"] = df["predicted_answer"] == df["answer"] 14 | total_accuracy = df["is_correct"].mean() 15 | 16 | print(f"Total Accuracy: {total_accuracy:.4f}") 17 | 18 | if "question_type" in df.columns: 19 | accuracy_report = df.groupby("question_type")["is_correct"].mean() 20 | print(accuracy_report) 21 | df["prefix"] = df["question_type"].apply(lambda x: x[0]) 22 | grouped_accuracy = df.groupby("prefix")["is_correct"].mean() 23 | print(grouped_accuracy) 24 | 25 | 26 | def map_prediction_to_answer_v2(row): 27 | answer_column = None 28 | if isinstance(row["pred"], str): 29 | prediction_letter = row["pred"][0] 30 | if prediction_letter in ["A", "B", "C", "D", "E"]: 31 | answer_column = "a" + str(ord(prediction_letter) - ord("A")) 32 | if "answer is " in row["pred"]: 33 | row["pred"] = row["pred"][row["pred"].index("answer is") :] 34 | if "A:" in row["pred"] or "A)" in row["pred"]: 35 | answer_column = "a0" 36 | elif "B:" in row["pred"] or "B)" in row["pred"]: 37 | answer_column = "a1" 38 | elif "C:" in row["pred"] or "C)" in row["pred"]: 39 | answer_column = "a2" 40 | elif "D:" in row["pred"] or "D)" in row["pred"]: 41 | answer_column = "a3" 42 | elif "E:" in row["pred"] or "E)" in row["pred"]: 43 | answer_column = "a4" 44 | if answer_column in ["a0", "a1", "a2", "a3", "a4"]: 45 | return row[answer_column] 46 | elif answer_column: 47 | print(prediction_letter) 48 | return "None" 49 | 50 | 51 | def eval_multiple_choice_gpt4(df): 52 | df["predicted_answer"] = df.apply(map_prediction_to_answer_gpt4, axis=1) 53 | df["is_correct"] = df["predicted_answer"] == df["answer"] 54 | total_accuracy = df["is_correct"].mean() 55 | 56 | print(f"Total Accuracy: {total_accuracy:.4f}") 57 | if "question_type" in df.columns: 58 | accuracy_report = df.groupby("question_type")["is_correct"].mean() 59 | print(accuracy_report) 60 | df["prefix"] = df["question_type"].apply(lambda x: x[0]) 61 | grouped_accuracy = df.groupby("prefix")["is_correct"].mean() 62 | print(grouped_accuracy) 63 | 64 | 65 | def map_prediction_to_answer_gpt4(row): 66 | answer_column = None 67 | if isinstance(row["pred"], str): 68 | row["pred"] = row["pred"].replace(".", "") 69 | prediction_letter = row["pred"][0] 70 | if prediction_letter in ["0", "1", "2", "3", "4"]: 71 | return row["a" + str(prediction_letter)] 72 | return "" 73 | return row["a" + str(int(float(row["pred"])))] 74 | -------------------------------------------------------------------------------- /evaluation/gpt3_consistency_utils.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (c) 2024, Deep Representation Learning Research Group, Seoul National University. 3 | All rights reserved. 4 | SPDX-License-Identifier: BSD-3-Clause 5 | For full license text, see the LICENSE file in the repo root or https://opensource.org/licenses/BSD-3-Clause 6 | """ 7 | 8 | import math 9 | import os 10 | import glob 11 | from tqdm import tqdm 12 | import openai 13 | import pandas as pd 14 | 15 | 16 | def eval_gpt3_consistency(df_merged1, df_merged2, path_result, api_key): 17 | 18 | os.makedirs(path_result, exist_ok=True) 19 | 20 | for (idx1, row1), (idx2, row2) in zip(df_merged1.iterrows(), df_merged2.iterrows()): 21 | process_gpt3_evaluation_consistency(row1, row2, path_result, api_key) 22 | 23 | result_path = os.path.join(path_result, "result.csv") 24 | 25 | if not os.path.exists(result_path): 26 | df_qa, path_merged = merge_qa_and_answer_consistency( 27 | df_merged1, df_merged2, path_result 28 | ) 29 | return df_qa, path_merged 30 | else: 31 | path_merged_already = result_path 32 | df_already = pd.read_csv(path_merged_already, index_col=0) 33 | return df_already, path_merged_already 34 | 35 | 36 | def process_gpt3_evaluation_consistency(row1, row2, path_result, api_key): 37 | client = openai.OpenAI(api_key=api_key) 38 | file_path_saved = os.path.join(path_result, str(row1["question_id"]) + ".txt") 39 | if not os.path.exists(file_path_saved): 40 | question1 = row1["question"] 41 | question2 = row2["question"] 42 | answer = row1["answer"] 43 | pred1 = row1["pred"] 44 | pred2 = row2["pred"] 45 | message = [ 46 | { 47 | "role": "system", 48 | "content": "You are an intelligent chatbot designed for evaluating the consistency of generative outputs for similar video-based question-answer pairs. " 49 | "You will be given two very similar questions, a common answer common to both the questions and predicted answers for the two questions ." 50 | "Your task is to compare the predicted answers for two very similar question, with a common correct answer and determine if they are consistent. Here's how you can accomplish the task:" 51 | "------" 52 | "##INSTRUCTIONS: " 53 | "- Focus on the consistency between the two predicted answers and the correct answer. Both predicted answers should correspond to the correct answer and to each other, and should not contain any contradictions or significant differences in the conveyed information.\n" 54 | "- Both predicted answers must be consistent with each other and the correct answer, in terms of the information they provide about the video content.\n" 55 | "- Consider synonyms or paraphrases as valid matches, but only if they maintain the consistency in the conveyed information.\n" 56 | "- Evaluate the consistency of the two predicted answers compared to the correct answer.", 57 | }, 58 | { 59 | "role": "user", 60 | "content": "Please evaluate the following video-based question-answer pair:\n\n" 61 | f"Question 1: {question1}\n" 62 | f"Question 2: {question2}\n" 63 | f"Correct Answer: {answer}\n" 64 | f"Predicted Answer to Question 1: {pred1}\n" 65 | f"Predicted Answer to Question 2: {pred2}\n\n" 66 | "Provide your evaluation only as a consistency score where the consistency score is an integer value between 0 and 5, with 5 indicating the highest level of consistency. " 67 | "Please generate the response in the form of a Python dictionary string with keys 'score', where its value is the consistency score in INTEGER, not STRING." 68 | "DO NOT PROVIDE ANY OTHER OUTPUT TEXT OR EXPLANATION. Only provide the Python dictionary string. " 69 | "For example, your response should look like this: {''score': 4.8}.", 70 | }, 71 | ] 72 | completion = client.chat.completions.create( 73 | model="gpt-3.5-turbo", messages=message 74 | ) 75 | response_message = completion.choices[0].message.content 76 | 77 | with open(file_path_saved, "w") as f: 78 | f.write(response_message) 79 | else: 80 | print("exist") 81 | 82 | 83 | def merge_qa_and_answer_consistency(df_qa1, df_qa2, path_result): 84 | df_qa = df_qa1.copy() # Copy df_qa1 to create df_qa 85 | 86 | # Rename columns in df_qa 87 | df_qa.rename(columns={"question": "question1", "pred": "pred1"}, inplace=True) 88 | df_qa["gpt3_score"] = None 89 | df_qa["question2"] = df_qa2["question"] 90 | df_qa["pred2"] = df_qa2["pred"] 91 | 92 | for (idx1, row1), (idx2, row2) in zip(df_qa1.iterrows(), df_qa2.iterrows()): 93 | file_path_saved = path_result + str(row1["question_id"]) + ".txt" 94 | 95 | if os.path.exists(file_path_saved): 96 | with open(file_path_saved, "r") as file: 97 | try: 98 | file_contents = file.read() 99 | if file_contents.endswith("."): 100 | file_contents = file_contents[:-1] 101 | content_dict = eval(file_contents) 102 | 103 | df_qa.loc[idx1, "gpt3_score"] = content_dict["score"] 104 | except Exception as e: 105 | print(e) 106 | print(file_path_saved) 107 | continue 108 | else: 109 | print(file_path_saved + " not exist") 110 | 111 | path_merged = os.path.join(path_result, "result.csv") 112 | df_qa.to_csv(path_merged) 113 | 114 | print(df_qa["gpt3_score"].describe()) 115 | 116 | return df_qa, path_merged 117 | -------------------------------------------------------------------------------- /evaluation/gpt3_evaluation_utils.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (c) 2024, Deep Representation Learning Research Group, Seoul National University. 3 | All rights reserved. 4 | SPDX-License-Identifier: BSD-3-Clause 5 | For full license text, see the LICENSE file in the repo root or https://opensource.org/licenses/BSD-3-Clause 6 | """ 7 | 8 | import math 9 | import os 10 | import glob 11 | from tqdm import tqdm 12 | import openai 13 | import pandas as pd 14 | 15 | from pipeline_processor.record import * 16 | 17 | 18 | def eval_gpt3(df_merged, path_result, api_key, gpt_eval_type=EvaluationType.DEFAULT): 19 | 20 | os.makedirs(path_result, exist_ok=True) 21 | 22 | for idx, row in df_merged.iterrows(): 23 | process_gpt3_evaluation_v2( 24 | row, path_result, api_key, gpt_eval_type=gpt_eval_type 25 | ) 26 | 27 | result_path = os.path.join(path_result, "result.csv") 28 | 29 | if not os.path.exists(result_path): 30 | df_qa, path_merged = merge_qa_and_answer(df_merged, path_result) 31 | return df_qa, path_merged 32 | else: 33 | path_merged_already = result_path 34 | df_already = pd.read_csv(path_merged_already, index_col=0) 35 | return df_already, path_merged_already 36 | 37 | 38 | def process_gpt3_evaluation_v2( 39 | row, path_result, api_key, gpt_eval_type=EvaluationType.DEFAULT 40 | ): 41 | client = openai.OpenAI( 42 | api_key=api_key, 43 | ) 44 | file_path_saved = os.path.join(path_result, str(row["question_id"]) + ".txt") 45 | if not os.path.exists(file_path_saved): 46 | question = row["question"] 47 | answer = row["answer"] 48 | pred = row["pred"] 49 | message = make_messages(question, answer, pred, gpt_eval_type) 50 | completion = client.chat.completions.create( 51 | model="gpt-3.5-turbo", messages=message 52 | ) 53 | # Convert response to a Python dictionary. 54 | response_message = completion.choices[0].message.content 55 | with open(file_path_saved, "w") as f: 56 | f.write(response_message) 57 | else: 58 | print("exist") 59 | 60 | 61 | def merge_qa_and_answer(df_qa, path_result): 62 | df_qa["gpt3_pred"] = None 63 | df_qa["gpt3_score"] = None 64 | 65 | for idx, row in df_qa.iterrows(): 66 | file_path_saved = os.path.join(path_result, str(row["question_id"]) + ".txt") 67 | 68 | if os.path.exists(file_path_saved): 69 | with open(file_path_saved, "r") as file: 70 | try: 71 | file_contents = file.read() 72 | if file_contents.endswith("."): 73 | file_contents = file_contents[:-1] 74 | content_dict = eval(file_contents) 75 | 76 | if "pred" in content_dict: 77 | df_qa.loc[idx, "gpt3_pred"] = content_dict["pred"] 78 | else: 79 | df_qa.loc[idx, "gpt3_pred"] = "" 80 | 81 | df_qa.loc[idx, "gpt3_score"] = content_dict["score"] 82 | except Exception as e: 83 | print(e) 84 | print(file_path_saved) 85 | continue 86 | else: 87 | print(file_path_saved + " not exist") 88 | 89 | path_merged = os.path.join(path_result, "result.csv") 90 | df_qa.to_csv(path_merged) 91 | 92 | yes_count = df_qa[df_qa["gpt3_pred"] == "yes"].shape[0] 93 | print(yes_count / df_qa.shape[0]) 94 | print(df_qa["gpt3_score"].describe()) 95 | 96 | return df_qa, path_merged 97 | 98 | 99 | def make_messages(question, answer, pred, gpt_eval_type): 100 | if gpt_eval_type == EvaluationType.DEFAULT: # default 101 | return [ 102 | { 103 | "role": "system", 104 | "content": "You are an intelligent chatbot designed for evaluating the correctness of generative outputs for question-answer pairs. " 105 | "Your task is to compare the predicted answer with the correct answer and determine if they match meaningfully. Here's how you can accomplish the task:" 106 | "------" 107 | "##INSTRUCTIONS: " 108 | "- Focus on the meaningful match between the predicted answer and the correct answer.\n" 109 | "- Consider synonyms or paraphrases as valid matches.\n" 110 | "- Evaluate the correctness of the prediction compared to the answer.", 111 | }, 112 | { 113 | "role": "user", 114 | "content": "Please evaluate the following video-based question-answer pair:\n\n" 115 | f"Question: {question}\n" 116 | f"Correct Answer: {answer}\n" 117 | f"Predicted Answer: {pred}\n\n" 118 | "Provide your evaluation only as a yes/no and score where the score is an integer value between 0 and 5, with 5 indicating the highest meaningful match. " 119 | "Please generate the response in the form of a Python dictionary string with keys 'pred' and 'score', where value of 'pred' is a string of 'yes' or 'no' and value of 'score' is in INTEGER, not STRING." 120 | "DO NOT PROVIDE ANY OTHER OUTPUT TEXT OR EXPLANATION. Only provide the Python dictionary string. " 121 | "For example, your response should look like this: {'pred': 'yes', 'score': 4.8}.", 122 | }, 123 | ] 124 | elif gpt_eval_type == EvaluationType.CORRECTNESS: 125 | return [ 126 | { 127 | "role": "system", 128 | "content": "You are an intelligent chatbot designed for evaluating the factual accuracy of generative outputs for video-based question-answer pairs. " 129 | "Your task is to compare the predicted answer with the correct answer and determine if they are factually consistent. Here's how you can accomplish the task:" 130 | "------" 131 | "##INSTRUCTIONS: " 132 | "- Focus on the factual consistency between the predicted answer and the correct answer. The predicted answer should not contain any misinterpretations or misinformation.\n" 133 | "- The predicted answer must be factually accurate and align with the video content.\n" 134 | "- Consider synonyms or paraphrases as valid matches.\n" 135 | "- Evaluate the factual accuracy of the prediction compared to the answer.", 136 | }, 137 | { 138 | "role": "user", 139 | "content": "Please evaluate the following video-based question-answer pair:\n\n" 140 | f"Question: {question}\n" 141 | f"Correct Answer: {answer}\n" 142 | f"Predicted Answer: {pred}\n\n" 143 | "Provide your evaluation only as a factual accuracy score where the factual accuracy score is an integer value between 0 and 5, with 5 indicating the highest level of factual consistency. " 144 | "Please generate the response in the form of a Python dictionary string with keys 'score', where its value is the factual accuracy score in INTEGER, not STRING." 145 | "DO NOT PROVIDE ANY OTHER OUTPUT TEXT OR EXPLANATION. Only provide the Python dictionary string. " 146 | "For example, your response should look like this: {''score': 4.8}.", 147 | }, 148 | ] 149 | elif gpt_eval_type == EvaluationType.DETAILED_ORIENTATION: 150 | return [ 151 | { 152 | "role": "system", 153 | "content": "You are an intelligent chatbot designed for evaluating the detail orientation of generative outputs for video-based question-answer pairs. " 154 | "Your task is to compare the predicted answer with the correct answer and determine its level of detail, considering both completeness and specificity. Here's how you can accomplish the task:" 155 | "------" 156 | "##INSTRUCTIONS: " 157 | "- Check if the predicted answer covers all major points from the video. The response should not leave out any key aspects.\n" 158 | "- Evaluate whether the predicted answer includes specific details rather than just generic points. It should provide comprehensive information that is tied to specific elements of the video.\n" 159 | "- Consider synonyms or paraphrases as valid matches.\n" 160 | "- Provide a single evaluation score that reflects the level of detail orientation of the prediction, considering both completeness and specificity.", 161 | }, 162 | { 163 | "role": "user", 164 | "content": "Please evaluate the following video-based question-answer pair:\n\n" 165 | f"Question: {question}\n" 166 | f"Correct Answer: {answer}\n" 167 | f"Predicted Answer: {pred}\n\n" 168 | "Provide your evaluation only as a detail orientation score where the detail orientation score is an integer value between 0 and 5, with 5 indicating the highest level of detail orientation. " 169 | "Please generate the response in the form of a Python dictionary string with keys 'score', where its value is the detail orientation score in INTEGER, not STRING." 170 | "DO NOT PROVIDE ANY OTHER OUTPUT TEXT OR EXPLANATION. Only provide the Python dictionary string. " 171 | "For example, your response should look like this: {''score': 4.8}.", 172 | }, 173 | ] 174 | elif gpt_eval_type == EvaluationType.CONTEXT: 175 | return [ 176 | { 177 | "role": "system", 178 | "content": "You are an intelligent chatbot designed for evaluating the contextual understanding of generative outputs for video-based question-answer pairs. " 179 | "Your task is to compare the predicted answer with the correct answer and determine if the generated response aligns with the overall context of the video content. Here's how you can accomplish the task:" 180 | "------" 181 | "##INSTRUCTIONS: " 182 | "- Evaluate whether the predicted answer aligns with the overall context of the video content. It should not provide information that is out of context or misaligned.\n" 183 | "- The predicted answer must capture the main themes and sentiments of the video.\n" 184 | "- Consider synonyms or paraphrases as valid matches.\n" 185 | "- Provide your evaluation of the contextual understanding of the prediction compared to the answer.", 186 | }, 187 | { 188 | "role": "user", 189 | "content": "Please evaluate the following video-based question-answer pair:\n\n" 190 | f"Question: {question}\n" 191 | f"Correct Answer: {answer}\n" 192 | f"Predicted Answer: {pred}\n\n" 193 | "Provide your evaluation only as a contextual understanding score where the contextual understanding score is an integer value between 0 and 5, with 5 indicating the highest level of contextual understanding. " 194 | "Please generate the response in the form of a Python dictionary string with keys 'score', where its value is contextual understanding score in INTEGER, not STRING." 195 | "DO NOT PROVIDE ANY OTHER OUTPUT TEXT OR EXPLANATION. Only provide the Python dictionary string. " 196 | "For example, your response should look like this: {''score': 4.8}.", 197 | }, 198 | ] 199 | elif gpt_eval_type == EvaluationType.TEMPORAL: 200 | return [ 201 | { 202 | "role": "system", 203 | "content": "You are an intelligent chatbot designed for evaluating the temporal understanding of generative outputs for video-based question-answer pairs. " 204 | "Your task is to compare the predicted answer with the correct answer and determine if they correctly reflect the temporal sequence of events in the video content. Here's how you can accomplish the task:" 205 | "------" 206 | "##INSTRUCTIONS: " 207 | "- Focus on the temporal consistency between the predicted answer and the correct answer. The predicted answer should correctly reflect the sequence of events or details as they are presented in the video content.\n" 208 | "- Consider synonyms or paraphrases as valid matches, but only if the temporal order is maintained.\n" 209 | "- Evaluate the temporal accuracy of the prediction compared to the answer.", 210 | }, 211 | { 212 | "role": "user", 213 | "content": "Please evaluate the following video-based question-answer pair:\n\n" 214 | f"Question: {question}\n" 215 | f"Correct Answer: {answer}\n" 216 | f"Predicted Answer: {pred}\n\n" 217 | "Provide your evaluation only as a temporal accuracy score where the temporal accuracy score is an integer value between 0 and 5, with 5 indicating the highest level of temporal consistency. " 218 | "Please generate the response in the form of a Python dictionary string with keys 'score', where its value is the temporal accuracy score in INTEGER, not STRING." 219 | "DO NOT PROVIDE ANY OTHER OUTPUT TEXT OR EXPLANATION. Only provide the Python dictionary string. " 220 | "For example, your response should look like this: {''score': 4.8}.", 221 | }, 222 | ] 223 | -------------------------------------------------------------------------------- /example/QzulzuWr0Zw_20_45.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/example/QzulzuWr0Zw_20_45.mp4 -------------------------------------------------------------------------------- /example/extraction_sample/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/example/extraction_sample/.DS_Store -------------------------------------------------------------------------------- /example/extraction_sample/rlQ2kW-FvMk_66_79/1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/example/extraction_sample/rlQ2kW-FvMk_66_79/1.jpg -------------------------------------------------------------------------------- /example/extraction_sample/rlQ2kW-FvMk_66_79/2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/example/extraction_sample/rlQ2kW-FvMk_66_79/2.jpg -------------------------------------------------------------------------------- /example/extraction_sample/rlQ2kW-FvMk_66_79/3.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/example/extraction_sample/rlQ2kW-FvMk_66_79/3.jpg -------------------------------------------------------------------------------- /example/extraction_sample/rlQ2kW-FvMk_66_79/4.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/example/extraction_sample/rlQ2kW-FvMk_66_79/4.jpg -------------------------------------------------------------------------------- /example/extraction_sample/rlQ2kW-FvMk_66_79/5.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/example/extraction_sample/rlQ2kW-FvMk_66_79/5.jpg -------------------------------------------------------------------------------- /example/extraction_sample/rlQ2kW-FvMk_66_79/6.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/example/extraction_sample/rlQ2kW-FvMk_66_79/6.jpg -------------------------------------------------------------------------------- /example/extraction_sample/ysTmUTQ5wZE_17_45/1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/example/extraction_sample/ysTmUTQ5wZE_17_45/1.jpg -------------------------------------------------------------------------------- /example/extraction_sample/ysTmUTQ5wZE_17_45/2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/example/extraction_sample/ysTmUTQ5wZE_17_45/2.jpg -------------------------------------------------------------------------------- /example/extraction_sample/ysTmUTQ5wZE_17_45/3.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/example/extraction_sample/ysTmUTQ5wZE_17_45/3.jpg -------------------------------------------------------------------------------- /example/extraction_sample/ysTmUTQ5wZE_17_45/4.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/example/extraction_sample/ysTmUTQ5wZE_17_45/4.jpg -------------------------------------------------------------------------------- /example/extraction_sample/ysTmUTQ5wZE_17_45/5.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/example/extraction_sample/ysTmUTQ5wZE_17_45/5.jpg -------------------------------------------------------------------------------- /example/extraction_sample/ysTmUTQ5wZE_17_45/6.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/example/extraction_sample/ysTmUTQ5wZE_17_45/6.jpg -------------------------------------------------------------------------------- /example/imagegrid_sample/QzulzuWr0Zw_20_45.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/example/imagegrid_sample/QzulzuWr0Zw_20_45.jpg -------------------------------------------------------------------------------- /example/imagegrid_sample/rlQ2kW-FvMk_66_79.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/example/imagegrid_sample/rlQ2kW-FvMk_66_79.jpg -------------------------------------------------------------------------------- /example/rlQ2kW-FvMk_66_79.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/example/rlQ2kW-FvMk_66_79.mp4 -------------------------------------------------------------------------------- /example/ysTmUTQ5wZE_17_45.mp4: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/example/ysTmUTQ5wZE_17_45.mp4 -------------------------------------------------------------------------------- /model_processor/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/model_processor/.DS_Store -------------------------------------------------------------------------------- /model_processor/__init__.py: -------------------------------------------------------------------------------- 1 | # from llava_model_processor import LlavaProcessor 2 | # from base_model_inference import BaseModelInference 3 | # from videollava_model_processor import VideoLLavaProcessor 4 | -------------------------------------------------------------------------------- /model_processor/base_model_inference.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (c) 2024, Deep Representation Learning Research Group, Seoul National University. 3 | All rights reserved. 4 | SPDX-License-Identifier: BSD-3-Clause 5 | For full license text, see the LICENSE file in the repo root or https://opensource.org/licenses/BSD-3-Clause 6 | """ 7 | 8 | import os 9 | from abc import ABC, abstractmethod 10 | from enum import Enum 11 | 12 | 13 | class BaseModelInference(ABC): 14 | def __init__(self, model_name, local_save_path): 15 | self.model_name = model_name 16 | self.local_save_path = local_save_path 17 | self.error_list = [] 18 | 19 | @abstractmethod 20 | def load_model(self, **kwargs): 21 | pass 22 | 23 | @abstractmethod 24 | def inference(self, **kwargs): 25 | pass 26 | 27 | @abstractmethod 28 | def extract_answers(self): 29 | pass 30 | 31 | def save_local_file(self, answer): 32 | directory = os.path.dirname(self.local_save_path) 33 | if not os.path.exists(directory): 34 | os.makedirs(directory) 35 | 36 | with open(self.local_save_path, "w") as file: 37 | file.write(answer) 38 | 39 | def infer_and_save(self, **kwargs): 40 | try: 41 | self.inference(**kwargs) 42 | answer = self.extract_answers() 43 | return answer 44 | except Exception as e: 45 | self.error_list.append(e) 46 | print(e) 47 | return -1 48 | -------------------------------------------------------------------------------- /model_processor/gpt4_model_processor.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (c) 2024, Deep Representation Learning Research Group, Seoul National University. 3 | All rights reserved. 4 | SPDX-License-Identifier: BSD-3-Clause 5 | For full license text, see the LICENSE file in the repo root or https://opensource.org/licenses/BSD-3-Clause 6 | """ 7 | 8 | from .base_model_inference import * 9 | import requests 10 | import sys, os 11 | from PIL import Image 12 | 13 | 14 | class GPT4Inference(BaseModelInference): 15 | def __init__(self, model_name, local_save_path=""): 16 | super().__init__(model_name, local_save_path) 17 | 18 | def load_model(self, **kwargs): 19 | self.api_key = kwargs["api_key"] 20 | self.header = self._make_headers() 21 | 22 | def inference(self, *args, **kwargs): 23 | self._extract_arguments(**kwargs) 24 | payload = self._make_payload() 25 | self.response = self._request_gpt_api(payload) 26 | 27 | def _request_gpt_api(self, payload): 28 | response = requests.post( 29 | "https://api.openai.com/v1/chat/completions", 30 | headers=self.header, 31 | json=payload, 32 | ) 33 | return response.json() 34 | 35 | def extract_answers(self): 36 | if "error" in self.response: 37 | raise Exception("error on gpt4 api" + str(self.response)) 38 | return self.response["choices"][0]["message"]["content"] 39 | 40 | def _make_headers(self): 41 | return { 42 | "Content-Type": "application/json", 43 | "Authorization": f"Bearer {self.api_key}", 44 | } 45 | 46 | def _make_payload(self): 47 | if "" == self.assistant_prompt: 48 | return { 49 | "model": "gpt-4-vision-preview", 50 | "messages": [ 51 | self._make_system_prompt(), 52 | self._make_user_prompt(), 53 | ], 54 | "max_tokens": self.max_tokens, 55 | } 56 | else: 57 | return { 58 | "model": "gpt-4-vision-preview", 59 | "messages": [ 60 | self._make_system_prompt(), 61 | self._make_user_prompt(), 62 | self._make_assistant_prompt(), 63 | ], 64 | "max_tokens": self.max_tokens, 65 | } 66 | 67 | def _make_system_prompt(self): 68 | return {"role": "system", "content": [self.system_prompt]} 69 | 70 | def _make_user_prompt(self): 71 | return { 72 | "role": "user", 73 | "content": [ 74 | {"type": "text", "text": self.user_prompt}, 75 | { 76 | "type": "image_url", 77 | "image_url": { 78 | "url": f"data:image/jpeg;base64,{self.base64_image}", 79 | "detail": "high", 80 | }, 81 | }, 82 | ], 83 | } 84 | 85 | def _make_assistant_prompt(self): 86 | return {"role": "assistant", "content": [self.assistant_prompt]} 87 | 88 | def _extract_arguments(self, **kwargs): 89 | self.system_prompt = kwargs["system_prompt"] 90 | self.user_prompt = kwargs["user_prompt"] 91 | self.assistant_prompt = kwargs["assistant_prompt"] 92 | self.base64_image = kwargs["base64_img"] 93 | self.max_tokens = kwargs.get("max_tokens", 500) 94 | -------------------------------------------------------------------------------- /model_processor/llava2_model_processor.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (c) 2024, Deep Representation Learning Research Group, Seoul National University. 3 | All rights reserved. 4 | SPDX-License-Identifier: BSD-3-Clause 5 | For full license text, see the LICENSE file in the repo root or https://opensource.org/licenses/BSD-3-Clause 6 | """ 7 | 8 | from .base_model_inference import * 9 | import math 10 | import re 11 | from io import BytesIO 12 | import torch 13 | import torch.nn.functional as F 14 | from llava.model.builder import load_pretrained_model 15 | from llava.constants import ( 16 | IMAGE_TOKEN_INDEX, 17 | DEFAULT_IM_START_TOKEN, 18 | DEFAULT_IM_END_TOKEN, 19 | IMAGE_PLACEHOLDER, 20 | ) 21 | from llava.conversation import conv_templates 22 | from llava.mm_utils import ( 23 | process_images, 24 | tokenizer_image_token, 25 | get_model_name_from_path, 26 | ) 27 | 28 | 29 | class Llava2Processor(BaseModelInference): 30 | def __init__(self, model_name, local_save_path=""): 31 | super().__init__(model_name, local_save_path) 32 | 33 | def load_model(self): 34 | model_name = get_model_name_from_path(self.model_name) 35 | ( 36 | self.tokenizer, 37 | self.model, 38 | self.image_processor, 39 | self.context_len, 40 | ) = load_pretrained_model( 41 | self.model_name, 42 | None, 43 | model_name, 44 | device=torch.cuda.current_device(), 45 | device_map="cuda", 46 | ) 47 | 48 | def inference(self, *args, **kwargs): 49 | self._extract_arguments(**kwargs) 50 | # Prepare images 51 | image_sizes = [self.raw_image.size] 52 | images_tensor = process_images( 53 | [self.raw_image], self.image_processor, self.model.config 54 | ).to(self.model.device, dtype=torch.float16) 55 | 56 | # Prepare input_ids 57 | input_ids = ( 58 | tokenizer_image_token( 59 | self.user_prompt, self.tokenizer, IMAGE_TOKEN_INDEX, return_tensors="pt" 60 | ) 61 | .unsqueeze(0) 62 | .to(self.model.device) 63 | ) 64 | 65 | # Generate output 66 | with torch.inference_mode(): 67 | output_ids = self.model.generate( 68 | input_ids, 69 | images=images_tensor, 70 | image_sizes=image_sizes, 71 | do_sample=False, 72 | temperature=1.0, 73 | top_p=0.9, 74 | num_beams=1, 75 | max_new_tokens=512, 76 | use_cache=True, 77 | ) 78 | 79 | # Decode and return outputs 80 | self.result_rext = self.tokenizer.batch_decode( 81 | output_ids, skip_special_tokens=True 82 | )[0].strip() 83 | 84 | def extract_answers(self): 85 | return self.result_rext.split("ASSISTANT:")[-1] 86 | 87 | def _extract_arguments(self, **kwargs): 88 | self.user_prompt = kwargs["user_prompt"] 89 | self.raw_image = kwargs["raw_image"] 90 | self.max_new_tokens = kwargs.get("max_new_tokens", 300) 91 | self.do_sample = kwargs.get("do_sample", False) 92 | self.temperature = kwargs.get("temperature", 1) 93 | -------------------------------------------------------------------------------- /pipeline_processor/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/pipeline_processor/__init__.py -------------------------------------------------------------------------------- /pipeline_processor/gpt4_pipeline.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (c) 2024, Deep Representation Learning Research Group, Seoul National University. 3 | All rights reserved. 4 | SPDX-License-Identifier: BSD-3-Clause 5 | For full license text, see the LICENSE file in the repo root or https://opensource.org/licenses/BSD-3-Clause 6 | """ 7 | 8 | import uuid 9 | import pandas as pd 10 | import os 11 | import sys 12 | from tqdm import tqdm 13 | 14 | sys.path.append(os.path.dirname(os.path.abspath(os.path.dirname(__file__)))) 15 | from model_processor.gpt4_model_processor import * 16 | from vision_processor.fps_gridview_processor import * 17 | from pipeline_processor.record import * 18 | 19 | 20 | class Gpt4Pipeline: 21 | def __init__(self, path_qa, path_video_file_format, dir="./gpt4_pipeline_result/"): 22 | self.path_qa = path_qa 23 | self.path_result = dir 24 | self.path_video_file_format = path_video_file_format 25 | self.error_video_name = [] 26 | self.make_video_file_list() 27 | 28 | def make_video_file_list(self): 29 | self._load_qa_file() 30 | self.df_qa["path_video"] = self.df_qa.apply( 31 | lambda x: (self.path_video_file_format % (x["video_name"])), axis=1 32 | ) 33 | 34 | def set_component( 35 | self, 36 | api_key, 37 | system_prompt, 38 | user_prompt, 39 | assistant_prompt, 40 | func_user_prompt=lambda prompt, row: prompt % (row["question"]), 41 | calculate_max_row=lambda x: round(math.sqrt(x)), 42 | frame_fixed_number=6, 43 | ): 44 | self.system_prompt = system_prompt 45 | self.user_prompt = user_prompt 46 | self.assistant_prompt = assistant_prompt 47 | self.func_user_prompt = func_user_prompt 48 | 49 | self.calculate_max_row = calculate_max_row 50 | self.frame_fixed_number = frame_fixed_number 51 | 52 | self.model = GPT4Inference("gpt4-inference") 53 | self.model.load_model(api_key=api_key) 54 | 55 | self.fps_data_processor = FpsDataProcessor( 56 | save_option=SaveOption.BASE64, 57 | calcualte_max_row=self.calculate_max_row, 58 | frame_fixed_number=self.frame_fixed_number, 59 | ) 60 | 61 | extra_dir = "ffn=%s/" % (str(self.frame_fixed_number),) 62 | 63 | self._make_directory(extra_dir) 64 | 65 | def do_pipeline(self): 66 | 67 | for idx, row in tqdm(self.df_qa.iterrows()): 68 | question_id = str(row["question_id"]) 69 | video_path = row["path_video"] 70 | video_extensions = ["avi", "mp4", "mkv", "webm", "gif"] 71 | 72 | if not os.path.exists(video_path): 73 | base_video_path, _ = os.path.splitext(video_path) 74 | for ext in video_extensions: 75 | temp_path = f"{base_video_path}.{ext}" 76 | if os.path.exists(temp_path): 77 | video_path = temp_path 78 | break 79 | 80 | if not os.path.exists(self._make_file_path(question_id)): 81 | try: 82 | image_data = self.fps_data_processor.process([video_path]) 83 | 84 | answer = self.model.infer_and_save( 85 | system_prompt=self.system_prompt, 86 | user_prompt=self.func_user_prompt(self.user_prompt, row), 87 | assistant_prompt=self.assistant_prompt, 88 | base64_img=image_data, 89 | ) 90 | 91 | if -1 != answer: 92 | self.write_result_file(question_id, answer) 93 | else: 94 | self.error_video_name.append(video_path) 95 | except Exception as e: 96 | print(e) 97 | print(video_path) 98 | continue 99 | return self.merge_qa_and_answer() 100 | 101 | def write_result_file(self, question_id, answer, extension=".txt"): 102 | file_path = self._make_file_path(question_id, extension) 103 | with open(file_path, "w") as file: 104 | file.write(answer) 105 | 106 | def _make_file_path(self, question_id, extension=".txt"): 107 | return os.path.join(self.path_result, question_id + extension) 108 | 109 | def _load_qa_file(self): 110 | self.df_qa = pd.read_csv(self.path_qa, index_col=0) 111 | 112 | def _make_directory(self, extra_dir): 113 | self.path_result = self.path_result + extra_dir 114 | os.makedirs(self.path_result, exist_ok=True) 115 | 116 | def merge_qa_and_answer(self): 117 | self.df_qa["pred"] = None 118 | if "answer_string" in self.df_qa.columns: 119 | # in case of NextQA 120 | self.df_qa = self.df_qa.drop("answer", axis=1).rename( 121 | {"answer_string": "answer"}, axis=1 122 | ) 123 | 124 | for idx, row in self.df_qa.iterrows(): 125 | question_id = str(row["question_id"]) 126 | file_path = self._make_file_path( 127 | question_id, 128 | ) 129 | 130 | if os.path.exists(file_path): 131 | with open(file_path, "r") as file: 132 | file_contents = file.read() 133 | self.df_qa.loc[idx, "pred"] = file_contents 134 | 135 | path_merged = os.path.join(self.path_result, "result.csv") 136 | self.df_qa.to_csv(path_merged) 137 | return self.df_qa, path_merged 138 | -------------------------------------------------------------------------------- /pipeline_processor/llava_pipeline.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (c) 2024, Deep Representation Learning Research Group, Seoul National University. 3 | All rights reserved. 4 | SPDX-License-Identifier: BSD-3-Clause 5 | For full license text, see the LICENSE file in the repo root or https://opensource.org/licenses/BSD-3-Clause 6 | """ 7 | 8 | import uuid 9 | import math 10 | import pandas as pd 11 | 12 | from tqdm import tqdm 13 | 14 | import os 15 | import sys 16 | 17 | sys.path.append(os.path.dirname(os.path.abspath(os.path.dirname(__file__)))) 18 | from model_processor.llava2_model_processor import * 19 | from vision_processor.fps_gridview_processor import * 20 | from .record import * 21 | 22 | 23 | class LlavaPipeline: 24 | def __init__( 25 | self, 26 | model_name, 27 | path_qa, 28 | path_video_file_format, 29 | dir="./llava_pipeline_result/", 30 | ): 31 | self.model_name = "liuhaotian/" + model_name 32 | self.path_qa = path_qa 33 | self.path_dir = dir 34 | self.path_result = dir 35 | self.path_video_file_format = path_video_file_format 36 | self.error_video_name = [] 37 | self.make_video_file_list() 38 | self.load_model() 39 | 40 | def make_video_file_list(self): 41 | self._load_qa_file() 42 | self.df_qa["path_video"] = self.df_qa.apply( 43 | lambda x: (self.path_video_file_format % (x["video_name"])), axis=1 44 | ) 45 | 46 | def load_model(self): 47 | self.model = Llava2Processor(self.model_name) 48 | self.model.load_model() 49 | 50 | def set_component( 51 | self, 52 | user_prompt, 53 | frame_fixed_number=6, 54 | func_user_prompt=lambda prompt, row: prompt % (row["question"]), 55 | calculate_max_row=lambda x: round(math.sqrt(x)), 56 | ): 57 | 58 | if not hasattr(self, "model"): 59 | raise AttributeError("Model is not loaded. Please call load_model()") 60 | 61 | self.frame_fixed_number = frame_fixed_number 62 | self.user_prompt = user_prompt 63 | self.func_user_prompt = func_user_prompt 64 | self.calculate_max_row = calculate_max_row 65 | 66 | self.fps_data_processor = FpsDataProcessor( 67 | save_option=SaveOption.IMAGE, 68 | calcualte_max_row=self.calculate_max_row, 69 | frame_fixed_number=self.frame_fixed_number, 70 | ) 71 | 72 | extra_dir = "ffn=%s/" % (str(self.frame_fixed_number)) 73 | self._make_directory(extra_dir) 74 | 75 | def do_pipeline(self): 76 | print("start pipeline") 77 | 78 | for idx, row in tqdm(self.df_qa.iterrows()): 79 | question_id = str(row["question_id"]) 80 | video_path = row["path_video"] 81 | ts = row["ts"] if "ts" in row else None 82 | video_extensions = ["avi", "mp4", "mkv", "webm", "gif"] 83 | 84 | if not os.path.exists(video_path): 85 | base_video_path, _ = os.path.splitext(video_path) 86 | for ext in video_extensions: 87 | temp_path = f"{base_video_path}.{ext}" 88 | if os.path.exists(temp_path): 89 | video_path = temp_path 90 | break 91 | 92 | if not os.path.exists(self._make_file_path(question_id)): 93 | try: 94 | image_data = self.fps_data_processor.process([video_path], ts) 95 | 96 | answer = self.model.infer_and_save( 97 | user_prompt=self.func_user_prompt(self.user_prompt, row), 98 | raw_image=image_data, 99 | ) 100 | if -1 != answer: 101 | self.write_result_file(question_id, answer) 102 | else: 103 | self.error_video_name.append(video_path) 104 | except Exception as e: 105 | print(e) 106 | print(video_path) 107 | continue 108 | 109 | return self.merge_qa_and_answer() 110 | 111 | def write_result_file(self, question_id, answer, extension=".txt"): 112 | file_path = self._make_file_path(question_id, extension) 113 | with open(file_path, "w") as file: 114 | file.write(answer) 115 | 116 | def _make_file_path(self, question_id, extension=".txt"): 117 | return os.path.join(self.path_result, question_id + extension) 118 | 119 | def _load_qa_file(self): 120 | try: 121 | self.df_qa = pd.read_csv(self.path_qa, index_col=0) 122 | except Exception as e: 123 | print(e) 124 | raise Exception("not valid qa files") 125 | 126 | def _make_directory(self, extra_dir): 127 | self.path_result = os.path.join(self.path_dir, extra_dir) 128 | os.makedirs(self.path_result, exist_ok=True) 129 | 130 | def merge_qa_and_answer(self): 131 | print("start merge_qa_and_answer") 132 | 133 | self.df_qa["pred"] = None 134 | path_merged = os.path.join(self.path_result, "result.csv") 135 | 136 | if not os.path.exists(path_merged): 137 | for idx, row in self.df_qa.iterrows(): 138 | question_id = str(row["question_id"]) 139 | file_path = self._make_file_path( 140 | question_id, 141 | ) 142 | if os.path.exists(file_path): 143 | try: 144 | with open(file_path, "r") as file: 145 | file_contents = file.read() 146 | self.df_qa.loc[idx, "pred"] = file_contents 147 | except Exception as e: 148 | print(file_path) 149 | raise (e) 150 | 151 | self.df_qa.to_csv(path_merged) 152 | else: 153 | self.df_qa = pd.read_csv(path_merged, index_col=0) 154 | 155 | return self.df_qa, path_merged 156 | -------------------------------------------------------------------------------- /pipeline_processor/record.py: -------------------------------------------------------------------------------- 1 | """ 2 | Copyright (c) 2024, Deep Representation Learning Research Group, Seoul National University. 3 | All rights reserved. 4 | SPDX-License-Identifier: BSD-3-Clause 5 | For full license text, see the LICENSE file in the repo root or https://opensource.org/licenses/BSD-3-Clause 6 | """ 7 | 8 | from enum import Enum 9 | 10 | 11 | class EvaluationType(Enum): 12 | DEFAULT = 0 13 | CORRECTNESS = 1 14 | DETAILED_ORIENTATION = 2 15 | CONTEXT = 3 16 | TEMPORAL = 4 17 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | tqdm 2 | moviepy 3 | opencv-python 4 | openai==1.14.0 5 | torch==2.2.0 6 | bitsandbytes==0.42.0 7 | flash_attn==2.5.3 8 | transformers==4.36.2 9 | transformers-stream-generator==0.0.4 10 | torchvision==0.17.0 11 | pytorchvideo @ git+https://github.com/facebookresearch/pytorchvideo.git@28fe037d212663c6a24f373b94cc5d478c8c1a1d -------------------------------------------------------------------------------- /vision_processor/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/imagegridworth/IG-VLM/00d11ab4f835d4930cd19376a2dc5f73b7206f5a/vision_processor/__init__.py -------------------------------------------------------------------------------- /vision_processor/base_frame_extractor.py: -------------------------------------------------------------------------------- 1 | import os 2 | import cv2 3 | from abc import ABC, abstractmethod 4 | import numpy as np 5 | from enum import Enum 6 | import os 7 | import sys 8 | 9 | sys.path.append(os.path.dirname(os.path.abspath(os.path.dirname(__file__)))) 10 | from .decorator_processor import * 11 | from .video_validation import VideoValidator 12 | 13 | 14 | class SaveOption(Enum): 15 | BYTES = "bytes" 16 | FILE = "file" 17 | BASE64 = "base64" 18 | NUMPY = "numpy" 19 | IMAGE = "image" 20 | 21 | 22 | class BaseFrameExtractor(ABC): 23 | def __init__(self, video_path_with_list): 24 | self.video_path = os.path.join(*video_path_with_list) 25 | self._check_video_valid() 26 | 27 | @abstractmethod 28 | def extract_frames(self, **kwargs): 29 | pass 30 | 31 | def _extract_frame_on_option(self, **kwargs): 32 | data = self.extract_frames(**kwargs) 33 | return data 34 | 35 | def save_data_based_on_option(self, option, filename=None, quality=95, **kwargs): 36 | data = self._extract_frame_on_option(**kwargs) 37 | if option == SaveOption.BYTES: 38 | return self._save_data(data) 39 | elif option == SaveOption.FILE: 40 | return self._save_data_to_file(data, filename, quality) 41 | elif option == SaveOption.BASE64: 42 | return self._save_data_to_base64(data) 43 | elif option == SaveOption.NUMPY: 44 | return np.array(data) 45 | else: 46 | raise ValueError("Invalid option: {}".format(option)) 47 | 48 | def _check_video_valid(self): 49 | VideoValidator(self.video_path) 50 | 51 | @save_to_bytes 52 | def _save_data(self, data): 53 | return data 54 | 55 | @save_to_file 56 | def _save_data_to_file(self, data, filename, quality): 57 | data = cv2.cvtColor(data, cv2.COLOR_BGR2RGB) 58 | cv2.imwrite(filename, data, [int(cv2.IMWRITE_JPEG_QUALITY), quality]) 59 | pass 60 | 61 | @save_to_base64 62 | def _save_data_to_base64(self, data): 63 | return data 64 | -------------------------------------------------------------------------------- /vision_processor/base_post_processor.py: -------------------------------------------------------------------------------- 1 | import os 2 | import cv2 3 | from abc import ABC, abstractmethod 4 | import numpy as np 5 | from enum import Enum 6 | 7 | from PIL import Image 8 | 9 | import os 10 | import sys 11 | 12 | sys.path.append(os.path.dirname(os.path.abspath(os.path.dirname(__file__)))) 13 | from .decorator_processor import * 14 | from .base_frame_extractor import * 15 | 16 | 17 | class BasePostProcessor(ABC): 18 | def __init__(self): 19 | pass 20 | 21 | @abstractmethod 22 | def post_process(self, **kwargs): 23 | pass 24 | 25 | def _post_processo_on_option(self, *args, **kwargs): 26 | data = self.post_process(*args, **kwargs) 27 | return data 28 | 29 | def post_process_based_on_options(self, option, *args, **kwargs): 30 | self.option = option 31 | data = self._post_processo_on_option(*args, **kwargs) 32 | if option == SaveOption.BASE64: 33 | return self._save_data_to_base64(data, quality=self.quality) 34 | elif option == SaveOption.IMAGE: 35 | return Image.fromarray(data) 36 | else: 37 | raise ValueError("Invalid option: {}".format(option)) 38 | 39 | @save_to_base64 40 | def _save_data_to_base64(self, data, quality=95): 41 | return data 42 | 43 | @save_to_file 44 | def _save_data_to_file(self, data, filename=None, quality=95): 45 | cv2.imwrite(filename, data, [int(cv2.IMWRITE_JPEG_QUALITY), quality]) 46 | pass 47 | -------------------------------------------------------------------------------- /vision_processor/decorator_processor.py: -------------------------------------------------------------------------------- 1 | import pickle 2 | import base64 3 | import os 4 | from io import BytesIO 5 | from PIL import Image 6 | 7 | 8 | def save_to_bytes(func): 9 | def wrapper(self, data): 10 | bytes_data = pickle.dumps(data) 11 | return func(self, bytes_data) 12 | 13 | return wrapper 14 | 15 | 16 | def save_to_one_file(func): 17 | def wrapper(self, data, filename): 18 | with open(filename, "wb") as file: 19 | func(self, data, file) 20 | 21 | return wrapper 22 | 23 | 24 | def save_to_file(func): 25 | def wrapper(self, data, filename, quality): 26 | os.makedirs(filename, exist_ok=True) 27 | for i, image_data in enumerate(data): 28 | file_path = os.path.join(filename, f"{i+1}.jpg") 29 | 30 | func(self, image_data, file_path, quality) 31 | 32 | return wrapper 33 | 34 | 35 | def save_to_base64(func): 36 | def wrapper(self, data, quality=95): 37 | rlt = Image.fromarray(data) 38 | 39 | with BytesIO() as byte_output: 40 | rlt.save(byte_output, format="JPEG", quality=quality) 41 | byte_output.seek(0) 42 | byte_data = byte_output.read() 43 | base64_data = base64.b64encode(byte_data).decode("utf-8") 44 | return func(self, base64_data) 45 | 46 | return wrapper 47 | -------------------------------------------------------------------------------- /vision_processor/fps_extractor.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | import math 4 | 5 | sys.path.append(os.path.dirname(os.path.abspath(os.path.dirname(__file__)))) 6 | 7 | from .base_frame_extractor import * 8 | from .video_validation import * 9 | from pipeline_processor.record import * 10 | from .gridview_generator import * 11 | 12 | 13 | class FpsExtractor(BaseFrameExtractor): 14 | def __init__(self, video_path): 15 | super().__init__(video_path) 16 | 17 | def __del__(self): 18 | self._release_video() 19 | 20 | def extract_frames(self, **kwargs): 21 | self._extract_arguments(**kwargs) 22 | self._open_video(self.ts) 23 | self.image_data, self.frame_count = self._process_video() 24 | return self.image_data 25 | 26 | def _open_video(self, ts=None): 27 | self.video_loader = VideoLoader(self.video_path) 28 | 29 | ( 30 | self.video_capture, 31 | self.fps, 32 | self.total_frames, 33 | self.video_length, 34 | ) = self.video_loader.open_video(ts=ts) 35 | 36 | def _release_video(self): 37 | self.video_loader.release_video() 38 | 39 | def _extract_arguments(self, **kwargs): 40 | try: 41 | self.frame_fixed_number = kwargs.get("frame_fixed_number", 6) 42 | self.ts = kwargs.get("ts", None) 43 | except Exception as e: 44 | raise Exception(e) 45 | 46 | def _process_video(self): 47 | frame_count = 0 48 | output_frame_index = 0 49 | list_frame_data = [] 50 | 51 | self._adjust_interval_fixed() 52 | 53 | frames = self.video_capture.iter_frames() 54 | 55 | for frame in frames: 56 | 57 | if frame_count % self.frames_per_interval == 0: 58 | start = output_frame_index * self.frames_per_interval 59 | frame_index_selected = start 60 | 61 | if frame_index_selected == frame_count: 62 | list_frame_data.append(frame) 63 | output_frame_index += 1 64 | 65 | if len(list_frame_data) == self.frame_fixed_number: 66 | break 67 | 68 | if frame_count == self.total_frames: 69 | break 70 | 71 | frame_count += 1 72 | 73 | return list_frame_data, output_frame_index 74 | 75 | def _adjust_interval_fixed(self): 76 | self.frames_per_interval = math.floor( 77 | self.total_frames / self.frame_fixed_number 78 | ) 79 | 80 | 81 | def main(): 82 | path = ["example", "ysTmUTQ5wZE_17_45.mp4"] 83 | tmp = FpsExtractor(path) 84 | print(tmp.video_path) 85 | tmp.save_data_based_on_option( 86 | SaveOption.FILE, 87 | filename="/example/extraction_sample/ysTmUTQ5wZE_17_45/", 88 | frame_fixed_number=6, 89 | ) 90 | 91 | 92 | if __name__ == "__main__": 93 | main() 94 | -------------------------------------------------------------------------------- /vision_processor/fps_gridview_processor.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import os 3 | import math 4 | import numpy as np 5 | 6 | sys.path.append(os.path.dirname(os.path.abspath(os.path.dirname(__file__)))) 7 | from pipeline_processor.record import * 8 | from .fps_extractor import * 9 | from .gridview_generator import * 10 | from .video_validation import * 11 | 12 | 13 | class FpsDataProcessor: 14 | def __init__( 15 | self, 16 | calcualte_max_row=lambda x: round(math.sqrt(x)), 17 | save_option=SaveOption.IMAGE, 18 | frame_fixed_number=6, 19 | ): 20 | self.calculate_max_row = calcualte_max_row 21 | self.frame_fixed_number = frame_fixed_number 22 | self.save_option = save_option 23 | 24 | def process(self, video_path, ts=None): 25 | fps_extractor = FpsExtractor(video_path) 26 | grid_view_creator = GridViewCreator( 27 | self.calculate_max_row, 28 | ) 29 | 30 | try: 31 | rlt_fps_extractor = fps_extractor.save_data_based_on_option( 32 | SaveOption.NUMPY, 33 | frame_fixed_number=self.frame_fixed_number, 34 | ts=ts, 35 | ) 36 | rlt_grid_view_creator = grid_view_creator.post_process_based_on_options( 37 | self.save_option, rlt_fps_extractor 38 | ) 39 | except Exception as e: 40 | print("Exception : %s on %s" % (str(e), str(video_path))) 41 | return -1 42 | 43 | return rlt_grid_view_creator 44 | 45 | 46 | def main(): 47 | 48 | video_name = "rlQ2kW-FvMk_66_79.mp4" 49 | 50 | fps_data_processor = FpsDataProcessor( 51 | save_option=SaveOption.IMAGE, 52 | frame_fixed_number=6, 53 | ) 54 | print(vars(fps_data_processor)) 55 | rlt = fps_data_processor.process(["example", video_name]) 56 | print(rlt) 57 | 58 | rlt.save("./example/imagegrid_sample/%s.jpg" % (video_name.split(".")[0])) 59 | 60 | 61 | if __name__ == "__main__": 62 | main() 63 | -------------------------------------------------------------------------------- /vision_processor/gridview_generator.py: -------------------------------------------------------------------------------- 1 | import cv2 2 | import glob 3 | import numpy as np 4 | import math 5 | import os 6 | import sys 7 | import random 8 | 9 | sys.path.append(os.path.dirname(os.path.abspath(os.path.dirname(__file__)))) 10 | from pipeline_processor.record import * 11 | from .base_post_processor import * 12 | from .fps_extractor import * 13 | 14 | 15 | class GridViewCreator(BasePostProcessor): 16 | def __init__(self, func_max_per_row, quality=95): 17 | self.func_max_per_row = func_max_per_row 18 | self.quality = quality 19 | 20 | def post_process(self, *args, **kwargs): 21 | return self.create_grid_view_as_array_from_image_array(*args) 22 | 23 | def _extract_arguments(self, **kwargs): 24 | try: 25 | self.images = kwargs.get("images", []) 26 | except Exception as e: 27 | raise Exception(e) 28 | 29 | def create_grid_view_as_array_from_image_array(self, images): 30 | images = images 31 | self.max_images_per_row = self.func_max_per_row(len(images)) 32 | 33 | min_width = min(img.shape[1] for img in images) 34 | min_height = min(img.shape[0] for img in images) 35 | resized_images = [cv2.resize(img, (min_width, min_height)) for img in images] 36 | 37 | while len(resized_images) % self.max_images_per_row != 0: 38 | resized_images.append( 39 | np.ones((min_height, min_width, 3), dtype=np.uint8) * 255 40 | ) 41 | 42 | image_rows = [ 43 | resized_images[i : i + self.max_images_per_row] 44 | for i in range(0, len(resized_images), self.max_images_per_row) 45 | ] 46 | concatenated_rows = [np.hstack(row) for row in image_rows] 47 | 48 | grid_image = np.vstack(concatenated_rows) 49 | image_array_rgb = cv2.cvtColor(grid_image, cv2.COLOR_BGR2RGB) 50 | if self.option == SaveOption.BASE64: 51 | return image_array_rgb 52 | else: 53 | return grid_image 54 | 55 | def _get_frame_number(self, file_path): 56 | numbers = file_path.split("/")[-1].split(".")[0] 57 | return int(numbers) if numbers else -1 58 | -------------------------------------------------------------------------------- /vision_processor/video_validation.py: -------------------------------------------------------------------------------- 1 | import os 2 | from moviepy.editor import VideoFileClip 3 | 4 | 5 | class VideoLoader: 6 | def __init__(self, video_path): 7 | self.video_path = video_path 8 | self.video_capture = None 9 | 10 | def file_exists(self): 11 | return os.path.isfile(self.video_path) 12 | 13 | def check_open_video(self): 14 | try: 15 | VideoFileClip(self.video_path) 16 | return True 17 | except Exception as e: 18 | print(f"Error opening video file: {self.video_path}. Reason: {str(e)}") 19 | return False 20 | 21 | def open_video(self, ts=None): 22 | try: 23 | self.video_capture = VideoFileClip(self.video_path) 24 | fps = self.video_capture.fps 25 | total_frames = int(self.video_capture.reader.nframes) 26 | video_length = total_frames / fps 27 | 28 | if ts is not None: 29 | start_time, end_time = ts.split("-") 30 | start_frame = int(float(start_time) * fps) 31 | end_frame = int(float(end_time) * fps) 32 | self.video_capture = self.video_capture.subclip(start_time, end_time) 33 | 34 | return ( 35 | self.video_capture, 36 | fps, 37 | end_frame - start_frame + 1, 38 | (end_frame - start_frame + 1) / fps, 39 | ) 40 | 41 | return self.video_capture, fps, total_frames, video_length 42 | 43 | except Exception as e: 44 | print(f"Error opening video file: {self.video_path}. Reason: {str(e)}") 45 | return False 46 | 47 | def release_video(self): 48 | del self.video_capture 49 | 50 | 51 | class VideoValidator: 52 | def __init__(self, video_path): 53 | self.video_path = video_path 54 | self.video_loader = VideoLoader(video_path) 55 | self.check_and_open_video() 56 | 57 | def check_and_open_video(self): 58 | if not self.video_loader.file_exists(): 59 | raise FileNotFoundError(f"File not found: {self.video_path}") 60 | 61 | if not self.video_loader.check_open_video(): 62 | raise Exception(f"Unable to open video file: {self.video_path}") 63 | --------------------------------------------------------------------------------