└── README.md


/README.md:
--------------------------------------------------------------------------------
 1 | # Erya
 2 | This repository is the official implementation of NLPCC 2023 paper: **Towards Effective Ancient Chinese Translation: Dataset, Model, and Evaluation**. 
 3 | 
 4 | The implementation is based on the text generation library [TextBox 2.0](https://github.com/RUCAIBox/TextBox).
 5 | 
 6 | ## Installation
 7 | You should clone the TextBox repository and follow its instructions.
 8 | ```
 9 | git clone https://github.com/RUCAIBox/TextBox.git && cd TextBox
10 | bash install.sh
11 | ```
12 | 
13 | ## Datasets
14 | 
15 | You can download Erya datasets in: https://huggingface.co/datasets/RUCAIBox/Erya-dataset. You should download datasets such as xint in it and place them in the `dataset` folder.
16 | 
17 | To be specific, the datasets in Erya benchmark and their corresponding title are:
18 | 
19 | - hans: Book of han
20 | - mings: Ming History
21 | - shij: Shi Ji
22 | - taip: Taiping Guangji
23 | - xint: New Tang History
24 | - xux: Xu Xiake's Travels
25 | 
26 | 
27 | ## Fine-tuning and Inference
28 | After setting up the environment, you can either use Erya model in the zero-shot scenario, or further tune Erya4FT model for a better translation performance.
29 | 
30 | ### Inference
31 | We have released Erya model in: [https://huggingface.co/RUCAIBox/Erya](https://huggingface.co/RUCAIBox/Erya), which you can use directly to generate translation as below.
32 | 
33 | ```
34 | from transformers import BertTokenizer, CPTForConditionalGeneration
35 | 
36 | tokenizer = BertTokenizer.from_pretrained("RUCAIBox/Erya")
37 | model = CPTForConditionalGeneration.from_pretrained("RUCAIBox/Erya")
38 | 
39 | input_ids = tokenizer("安世字子孺，少以父任为郎。", return_tensors='pt')
40 | input_ids.pop("token_type_ids")
41 | 
42 | pred_ids = model.generate(max_new_tokens=256, **input_ids)
43 | print(tokenizer.batch_decode(pred_ids, skip_special_tokens=True))
44 | ```
45 | 
46 | ### Tuning
47 | We also released Erya4FT model in: [https://huggingface.co/RUCAIBox/Erya4FT](https://huggingface.co/RUCAIBox/Erya4FT), which you can further tune on the translation dataset.
48 | 
49 | ```
50 | python run_textbox.py --model=CPT --dataset=[dataset] --model_path=RUCAIBox/Erya4FT --epochs=[epoch_nums]
51 | ```
52 | 
53 | 


--------------------------------------------------------------------------------