├── .gitignore ├── Readme.md ├── data └── qts_jinti_dummy │ └── input.txt ├── inspect_checkpoint.lua ├── lstm_rnn.lua ├── model ├── GRU.lua ├── LSTM.lua └── RNN.lua ├── sample.lua ├── sample_qts.lua ├── util ├── CharSplitLMMinibatchLoader.lua ├── LineSplitLoader.lua ├── OneHot.lua ├── PaddedLineSplitLoader.lua ├── misc.lua └── model_utils.lua └── yunjiao ├── YunjiaoLoader.lua ├── ping.txt └── ze.txt /.gitignore: -------------------------------------------------------------------------------- 1 | *.t7 2 | -------------------------------------------------------------------------------- /Readme.md: -------------------------------------------------------------------------------- 1 | A fork of [char-rnn](https://github.com/karpathy/char-rnn) that learns from line samples, excellent at producing structured poems. 2 | Demo at [http://zhengwy.com/make-tangshi/](http://zhengwy.com/make-tangshi/) 3 | See [my blog](zhengwy.com/neural-network-for-tangshi/) that explains some details. 4 | 5 | ## Train 6 | To train this is the easiest (the following command is what worked for me, smaller network will not work so well) 7 | ``` 8 | th lstm_rnn.lua -data_dir data/qts_jinti_dummy -batch_size 20 -max_epochs 25 -eval_val_every 0.5 -seq_length 25 -rnn_size 500 -print_every 10 -savemodel 1 -train_frac 0.9 -val_frac 0.1 -learning_rate_decay_after 5 -dropout 0.2 9 | ``` 10 | if you are running into out of memory issue, reduce the rnn_size or batch_size 11 | 12 | The models are saved along the way. Pick the one with the lowest error, not the last one. 13 | 14 | ## Data 15 | I only included a small sample of Quan-Tang-Shi. If you want the full dataset please google and retrieve it on your own. 16 | 17 | ## Sample 18 | To sample you have two options 19 | To use the sampling function from the original char-rnn 20 | ``` 21 | th sample.lua 22 | ``` 23 | This will produce a bunch of text, 24 | To Sample with Pingze rules and end-of-poem characters, use sample_qts.lua 25 | ``` 26 | th sample_qts.lua 27 | ``` 28 | 29 | Be warned, the Yunjiao module is not complete in terms of dictionary, so if you see this error 30 | ``` 31 | ./yunjiao/YunjiaoLoader.lua:150: out of range char 祚 32 | ``` 33 | That means you have to go to some Pingshuiyun site and find that character, and add it to the correct line in either ping.txt or ze.txt. One possible such site is [wikisource](https://zh.wikisource.org/zh/%E5%B9%B3%E6%B0%B4%E9%9F%BB) 34 | 35 | To see what the parameter means, read the original documentation at [char-rnn](https://github.com/karpathy/char-rnn) 36 | 37 | ##License 38 | MIT 39 | 40 | -------------------------------------------------------------------------------- /data/qts_jinti_dummy/input.txt: -------------------------------------------------------------------------------- 1 | 荡子戍辽东，连年信不通。尘生锦步障，花送玉屏风。只怨红颜改，宁辞玉簟空。系书春雁足，早晚到云中。 2 | 普天皆灭焰，匝地尽藏烟。不知何处火，来就客心然。 3 | 岁去红颜尽，愁来白发新。今朝开镜匣，疑是别逢人。 4 | 闻道成都酒，无钱亦可求。不知将几斗，销得此来愁。 5 | 春雪满空来，触处似花开。不知园里树，若个是真梅。 6 | 零落嗟残命，萧条托胜因。方烧三界火，遽洗六情尘。隔岭天花发，凌空月殿新。谁令乡国梦，终此学分身。 7 | 擢擢当轩竹，青青重岁寒。心贞徒见赏，箨小未成竿。 8 | 闻君庭竹咏，幽意岁寒多。叹息为冠小，良工将奈何。 9 | 去年离别雁初归，今夜裁缝萤已飞。征客近来音信断，不知何处寄寒衣。 10 | 畏途方万里，生涯近百年。不知将白首，何处入黄泉。 11 | 回波尔时酒卮，微臣职在箴规。侍宴既过三爵，喧哗窃恐非仪。 12 | 九陌连灯影，千门度月华。倾城出宝骑，匝路转香车。烂熳惟愁晓，周游不问家。更逢清管发，处处落梅花。 13 | 游客三江外，单栖百虑违。山川忆处近，形影梦中归。夜月明虚帐，秋风入捣衣。从来不惯别，况属雁南飞。 14 | 銮舆上碧天，翠帟拖晴烟。绝崿纡仙径，层岩敞御筵。云披丹凤阙，日下黑龙川。更睹南熏奏，流声入管弦。 15 | 令节重遨游，分镳应彩球。骖驔回上苑，蹀躞绕通沟。影就红尘没，光随赭汗流。赏阑清景暮，歌舞乐时休。 16 | 玉府凌三曜，金坛驻六龙。彩旒悬倒景，羽盖偃乔松。玄圃灵芝秀，华池瑞液浓。谬因沾舜渥，长愿奉尧封。 17 | 广化三边静，通烟四海安。还将膝下爱，特副域中欢。圣念飞玄藻，仙仪下白兰。日斜征盖没，归骑动鸣鸾。 18 | 銮舆羽驾直城隈，帐殿旌门此地开。皎洁灵潭图日月，参差画舸结楼台。波摇岸影随桡转，风送荷香逐酒来。愿奉圣情欢不极，长游云汉几昭回。 19 | 銮辂青旂下帝台，东郊上苑望春来。黄莺未解林间啭，红蕊先从殿里开。画阁条风初变柳，银塘曲水半含苔。欣逢睿藻光韶律，更促霞觞畏景催。 20 | 绿叶迎春绿，寒枝历岁寒。愿持柏叶寿，长奉万年欢。 21 | 王孙帝女下仙台，金榜珠帘入夜开。遽惜琼筵欢正洽，唯愁银箭晓相催。 22 | 鸣銮赫奕下重楼，羽盖逍遥向一丘。汉日唯闻白衣宠，唐年更睹赤松游。 23 | 闻君墨绶出丹墀，双舄飞来伫有期。寄谢铜街攀柳日，无忘粉署握兰时。 24 | 剪彩迎初候，攀条故写真。花随红意发，叶就绿情新。嫩色惊衔燕，轻香误采人。应为熏风拂，能令芳树春。 25 | 出震乘东陆，凭高御北辰。祥云应早岁，瑞雪候初旬。庭树千花发，阶蓂七叶新。幸承今日宴，长寿万年春。 26 | 青女三秋节，黄姑七日期。星桥度玉珮，云阁掩罗帷。河气通仙掖，天文入睿词。今宵望灵汉，应得见蛾眉。 27 | 秋豫凝仙览，宸游转翠华。呼鹰下鸟路，戏马出龙沙。紫菊宜新寿，丹萸辟旧邪。须陪长久宴，岁岁奉吹花。 28 | 出豫乘秋节，登高陟梵宫。皇心满尘界，佛迹现虚空。日月宜长寿，人天得大通。喜闻题宝偈，受记莫由同。 29 | 云物中京晓，天人外馆开。飞桥象河汉，悬榜学蓬莱。北阙临仙槛，南山送寿杯。一窥轮奂毕，惭恧栋梁材。 30 | 圣后经纶远，谋臣计画多。受降追汉策，筑馆许戎和。俗化乌孙垒，春生积石河。六龙今出饯，双鹤愿为歌。 31 | 皇情遍九垓，御辇驻昭回。路若随天转，人疑近日来。河看大禹凿，山见巨灵开。愿扈登封驾，常持荐寿杯。 32 | 云骖驱半景，星跸坐中天。国诞玄宗圣，家寻碧落仙。玉杯鸾荐寿，宝算鹤知年。一睹光华旦，欣承道德篇。 33 | 两揆光天秩，三朝奉帝熙。何言集大鸟，忽此丧元龟。坐叹公槐落，行闻宰树悲。壑舟今已去，宁有济川期。 34 | 宝契无为属圣人，雕舆出幸玩芳辰。平楼半入南山雾，飞阁旁临东墅春。夹路秾花千树发，垂轩弱柳万条新。处处风光今日好，年年愿奉属车尘。 35 | 主第岩扃驾鹊桥，天门阊阖降鸾镳。历乱旌旗转云树，参差台榭入烟霄。林间花杂平阳舞，谷里莺和弄玉箫。已陪沁水追欢日，行奉茅山访道朝。 36 | 六龙齐轸御朝曦，双鹢维舟下绿池。飞观仰看云外耸，浮桥直见海中移。灵泉巧凿天孙渚，孝笋能抽帝女枝。幸愿一生同草树，年年岁岁乐于斯。 37 | 器乏雕梁器，材非构厦材。但将千岁叶，常奉万年杯。 38 | 始见青云干律吕，俄逢瑞雪应阳春。今日回看上林树，梅花柳絮一时新。 39 | 廊庙心存岩壑中，銮舆瞩在灞城东。逍遥自在蒙庄子，汉主徒言河上公。 40 | 水面芙蓉秋已衰，繁条偏是著花迟。平明露滴垂红脸，似有朝愁暮落时。 41 | 红萼竞燃春苑曙，粉茸新吐御筵开。长年愿奉西王母，近侍惭无东朔才。 42 | 望幸三秋暮，登高九日初。朱旗巡汉苑，翠帟俯秦墟。宠极萸房遍，恩深菊酎馀。承欢何以答，万亿奉宸居。 43 | 天跸三乘启，星舆六辔行。登高凌宝塔，极目遍王城。神卫空中绕，仙歌云外清。重阳千万寿，率舞颂升平。 44 | 西郊窈窕凤凰台，北渚平明法驾来。匝地金声初度曲，周堂玉溜好传杯。湾路分游画舟转，岸门相向碧亭开。微臣此时承宴乐，仿佛疑从星汉回。 45 | 公主林亭地，清晨降玉舆。画桥飞渡水，仙阁迥临虚。新晴看蛱蝶，早夏摘芙蕖。文酒娱游盛，忻叨侍从馀。 46 | 龙骖晓入望春宫，正逢春雪舞春风。花光并在天文上，寒气行销御酒中。 47 | 重九临商节，登高出汉宫。正逢萸实满，还对菊花丛。霁云开就日，仙藻丽秋风。微臣预在镐，窃抃遂无穷。 48 | 沙界人王塔，金绳梵帝游。言从祇树赏，行玩菊丛秋。御酒调甘露，天花拂彩旒。尧年将佛日，同此庆时休。 49 | 诘旦重门闻警跸，传言太主奏山林。是日回舆罗万骑，此时欢喜赐千金。鹭羽凤箫参乐曲，荻园竹径接帷阴。手舞足蹈方无已，万年千岁奉薰琴。 50 | 金榜岧峣云里开，玉箫参差天际回。莫惊侧弁还归路，只为平阳歌舞催。 51 | 远目瞰秦坰，重阳坐灞亭。既开黄菊酒，还降紫微星。箫鼓谙仙曲，山河入画屏。幸兹陪宴喜，无以效丹青。 52 | 万乘临真境，重阳眺远空。慈云浮雁塔，定水映龙宫。宝铎含飙响，仙轮带日红。天文将瑞色，辉焕满寰中。 53 | 重九开秋节，得一动宸仪。金风飘菊蕊，玉露泫萸枝。睿览八纮外，天文七曜披。临深应在即，居高岂忘危。 54 | 蜂蚁屯夷落，熊罴逐汉飞。忘躯百战后，屈指一年归。厚眷纾天藻，深慈解御衣。兴酣歌舞出，朝野叹光辉。 55 | 梁园开胜景，轩驾动宸衷。早荷承湛露，修竹引薰风。九酝倾钟石，百兽协丝桐。小臣陪宴镐，献寿奉维嵩。 56 | 銮舆巡上苑，凤驾瞰层城。御座丹乌丽，宸居白鹤惊。玉旗萦桂叶，金杯泛菊英。九晨陪圣膳，万岁奉承明。 57 | 九秋光顺豫，重节霁良辰。登高识汉苑，问道侍轩臣。菊花浮秬鬯，萸房插缙绅。圣化边陲谧，长洲鸿雁宾。 58 | 重阳玉律应，万乘金舆出。风起韵虞弦，云开吐尧日。菊花浮圣酒，茱香挂衰质。欲知恩煦多，顺动观秋实。 59 | 重阳乘令序，四野开晴色。日月数初并，乾坤圣登极。菊黄迎酒泛，松翠凌霜直。游海难为深，负山徒倦力。 60 | 九日报仙家，三秋转岁华。呼鹰下鸟路，戏马出龙沙。簪挂丹萸蕊，杯浮紫菊花。所愿同微物，年年共辟邪。 61 | 红萼竞燃春苑曙，莑茸新吐御筵开。长年愿奉西王宴，近侍惭无东朔才。 62 | 千钟圣酒御筵披，六出祥英乱绕枝。即此神仙对琼圃，何烦辙迹向瑶池。 63 | 御气三秋节，登高九曲门。桂筵罗玉俎，菊醴溢芳樽。遵渚归鸿度，承云舞鹤鶱。微臣滥陪赏，空荷圣明恩。 64 | 时和素秋节，宸豫紫机关。鹤似闻琴至，人疑宴镐还。旷望临平野，潺湲俯暝湾。无因酬大德，空此愧崇班。 65 | 代邸东南龙跃泉，清漪碧浪远浮天。楼台影就波中出，日月光疑镜里悬。雁沼洄流成舜海，龟书荐社应尧年。大川既济惭为楫，报德空思奉细涓。 66 | 洪慈均动植，至德俯深玄。出豫从初地，登高适梵天。白云飞御藻，慧日暖皇编。别有秋原藿，长倾雨露缘。 67 | 玉辇移中禁，珠梯览四禅。重阶清汉接，飞窦紫霄悬。缀叶披天藻，吹花散御筵。无因銮跸暇，俱舞鹤林前。 68 | 鹦林花塔启，凤辇顺时游。重九昭皇庆，大千扬帝休。耆闍妙法阐，王舍睿文流。至德覃无极，小臣歌讵酬。 69 | 扈跸游玄地，陪仙瞰紫微。似迈铢衣劫，将同羽化飞。雕戈秋日丽，宝剑晓霜霏。献觞乘菊序，长愿奉天晖。 70 | 净境重阳节，仙游万乘来。插萸登鹫岭，把菊坐蜂台。十地祥云合，三天瑞景开。秋风词更远，窃抃乐康哉。 71 | 应节萸房满，初寒菊圃新。龙旗焕辰极，凤驾俨香闉。莲井偏宜夏，梅梁更若春。一忻陪雁塔，还似得天身。 72 | 九月从时豫，三乘为法开。中霄日天子，半座宝如来。摘果珠盘献，攀萸玉辇回。愿将尘露点，遥奉光明台。 73 | 山豫乘金节，飞文焕日宫。萸房开圣酒，杏苑被玄功。塔向三天迥，禅收八解空。叨恩奉兰藉，终愧洽薰风。 74 | 飞塔凌霄起，宸游一届焉。金壶新泛菊，宝座即披莲。就日摇香辇，凭云出梵天。祥氛与佳色，相伴杂炉烟。 75 | 宝地邻丹掖，香台瞰碧云。河山天外出，城阙树中分。睿藻兰英秀，仙杯菊蕊薰。愿将今日乐，长奉圣明君。 76 | 九秋霜景净，千门晓望通。仙游光御路，瑞塔迥凌空。菊彩扬尧日，萸香绕舜风。天文丽辰象，窃抃仰层穹。 77 | 雪尽铜驼路，花照石崇家。年光开柳色，池影泛云华。赏洽情方远，春归景未赊。欲知多暇日，尊酒渍澄霞。 78 | 瑞塔临初地，金舆幸上方。空边有清净，觉处无馨香。雨霁微尘敛，风秋定水凉。兹辰采仙菊，荐寿庆重阳。 79 | 主第簪裾出，王畿春照华。山亭一以眺，城阙带烟霞。横堤列锦帐，傍浦驻香车。欢娱属晦节，酩酊未还家。 80 | 公门袭汉环，主第称秦玉。池架祥鳣序，山吹鸣凤曲。拂席萝薜垂，回舟芰荷触。平阳妙舞处，日暮清歌续。 81 | 锦楫沙棠舰，罗带石榴裙。绿潭采荷芰，清江日稍曛。鱼鸟争唼喋，花叶相芬氲。不觉芳洲暮，菱歌处处闻。 82 | 汉室欢娱盛，魏国文雅遒。许史多暮宿，应陈从夜游。西园宴公子，北里召王侯。讵似将军猎，空嗟亭尉留。 83 | 颍川豪横客，咸阳轻薄儿。田窦方贵幸，赵李新相知。轩盖终朝集，笙竽此夜吹。黄金盈箧笥，白日忽西驰。 84 | 春殿猗兰美，仙阶柏树荣。地逢芳节应，时睹圣人生。月满增祥荚，天长发瑞灵。南山遥可献，常愿奉皇明。 85 | 下嫁戎庭远，和亲汉礼优。笳声出虏塞，箫曲背秦楼。贵主悲黄鹤，征人怨紫骝。皇情眷亿兆，割念俯怀柔。 86 | 御跸下都门，军麾出塞垣。长杨跨武骑，细柳接戎轩。睿曲风云动，边威鼓吹喧。坐帷将阃外，俱是报明恩。 87 | 涌霄开宝塔，倒影驻仙舆。雁子乘堂处，龙王起藏初。秋风圣主曲，佳气史官书。愿献重阳寿，承欢万岁馀。 88 | 日斜漳浦望，风起邺台寒。玉座平生晚，金尊妓吹阑。舞馀依帐泣，歌罢向陵看。萧索松风暮，愁烟入井阑。 89 | 汉将留边朔，遥遥岁序深。谁堪牧马思，正是胡笳吟。曲断关山月，声悲雨雪阴。传书问苏武，陵也独何心。 90 | 青柳映红颜，黄云蔽紫关。忽闻边使出，枝叶为君攀。舞腰愁欲断，春心望不还。风花滚成雪，罗绮乱斑斑。 91 | 征客向轮台，幽闺寂不开。音书秋雁断，机杼夜蛩催。虚幌风吹叶，闲阶露湿苔。自怜愁思影，常共月裴回。 92 | 琼殿含光映早轮，玉鸾严跸望初晨。池开冻水仙宫丽，树发寒花禁苑新。佳气裴回笼细网，残霙淅沥染轻尘。良时荷泽皆迎胜，穷谷晞阳犹未春。 93 | 晨跸凌高转翠旌，春楼望远背朱城。忽排花上游天苑，却坐云边看帝京。百草香心初罥蝶，千林嫩叶始藏莺。幸同葵藿倾阳早，愿比盘根应候荣。 94 | 欲转声犹涩，将飞羽未调。高风不借便，何处得迁乔。 95 | 百舌鸣高树，弄音无常则。借问声何烦，末俗不尚默。 96 | 灵具醉，杳熙熙。灵将往，眇禗禗。愿明德，吐正词。烂遗光，流祯祺。 97 | 星汉下天孙，车服降殊蕃。匣中词易切，马上曲虚繁。关塞移朱帐，风尘暗锦轩。箫声去日远，万里望河源。 98 | 旧许星车转，神京祖帐开。断烟伤别望，零雨送离杯。辞燕依空绕，宾鸿入听哀。分襟与秋气，日夕共悲哉。 99 | 郎官出宰赴伊瀍，征传駸駸灞水前。此时怅望新丰道，握手相看共黯然。 100 | 弦歌试宰日，城阙赏心违。北谢苍龙去，南随黄鹄飞。夏云海中出，吴山江上微。甿谣岂云远，从此庆缁衣。 101 | 於穆清庙，肃雍严祀。合福受釐，介以繁祉。 102 | 征马噪金珂，嫖姚向北河。绿苔行迹少，红粉泪痕多。宝屋粘花絮，银筝覆网罗。别君如昨日，青海雁频过。 103 | 望月思氛氲，朱衾懒更熏。春生翡翠帐，花点石榴裙。燕语时惊妾，莺啼转忆君。交河一万里，仍隔数重云。 104 | 殊类骄无长，王师示有征。中军才受律，妖寇已亡精。斩虏还遮塞，绥降更筑城。从来攻必克，天策振奇兵。 105 | 乾坤启圣吐龙泉，泉水年年胜一年。始看鱼跃方成海，即睹飞龙利在天。洲渚遥将银汉接，楼台直与紫微连。休气荣光恒不散，悬知此地是神仙。 106 | 初岁开韶月，田家喜载阳。晚晴摇水态，迟景荡山光。浦净渔舟远，花飞樵路香。自然成野趣，都使俗情忘。 107 | 朱绂临秦望，皇华赴洛桥。文章南渡越，书奏北归朝。树入江云尽，城衔海月遥。秋风将客思，川上晚萧萧。 108 | 闻道山阴会，仍为火忌辰。途中甘弃日，江上苦伤春。流水翻催泪，寒灰更伴人。丹心终不改，白发为谁新。 109 | 巴东三峡尽，旷望九江开。楚塞云中出，荆门水上来。鱼龙潜啸雨，凫雁动成雷。南国秋风晚，客思几悠哉。 110 | 鹦鹉殊姿致，鸾皇得比肩。常寻金殿里，每话玉阶前。贾谊才方达，扬雄老未迁。能言既有地，何惜为闻天。 111 | 朱门长不闭，亲友恣相过。年今将半百，不乐复如何。 112 | 避贤初罢相，乐圣且衔杯。为问门前客，今朝几个来。 113 | 方如行义，圆如用智。动如逞才，静如遂意。 114 | 於赫皇祖，昭明有融。惟文之德，惟武之功。河海静谧，车书混同。虔恭孝飨，穆穆玄风。 115 | 於穆文考，圣神昭彰。箫勺群慝，含光远方。万物茂遂，九夷宾王。愔愔云韶，德音不忘。 116 | 上灵眷命膺会昌，盛德殷荐叶辰良。景福降兮圣德远，玄化穆兮天历长。 117 | 穆穆我后，道应千龄。登三处大，得一居贞。礼惟崇德，乐以和声。百神仰止，天下文明。 118 | 闓阳播气，甄曜垂明。有赫圜宰，深仁曲成。日丽苍璧，烟开紫营。聿遵乾享，式降鸿祯。 119 | 钦惟大帝，载仰皇穹。始命田烛，爰启郊宫。云门骇听，雷鼓鸣空。神其介祀，景祚斯融。 120 | 八音斯奏，三献毕陈。宝祚惟永，晖光日新。 121 | 叠璧凝影皇坛路，编珠流彩帝郊前。已奏黄钟歌大吕，还符宝历祚昌年。 122 | 昔在炎运终，中华乱无象。酆郊赤乌见，邙山黑云上。大资下周车，禁暴开殷网。幽明同叶赞，鼎祚齐天壤。 123 | 歌奏毕兮礼献终，六龙驭兮神将升。明德感兮非黍稷，降福简兮祚休征。 124 | 蘋蘩礼著，黍稷诚微。音盈凤管，彩驻龙旗。洪歆式就，介福攸归。送乐有阕，灵驭遄飞。 125 | 天之历数归睿唐，顾惟菲德钦昊苍。撰吉日兮表殷荐，冀神鉴兮降闓阳。 126 | 恭临宝位，肃奉瑶图。恒思解网，每轸泣辜。德惭巢燧，化劣唐虞。期我良弼，式赞嘉谟。 127 | 得一流玄泽，通三御紫宸。远叶千龄运，遐销九域尘。绝瑞骈阗集，殊祥络绎臻。登年庆栖亩，稔岁贺盈囷。 128 | 悠哉广覆，大矣曲成。九玄著象，七曜贞明。珪璧是奠，酝酎斯盈。作乐崇德，爰畅咸英。 129 | 郊坛展敬，严配因心。孤竹箫管，空桑瑟琴。肃穆大礼，铿锵八音。恭惟上帝，希降灵歆。 130 | 九成爰奏，三献式陈。钦承景福，恭托明禋。 131 | 坤元光至德，柔训阐皇风。芣苡芳声远，螽斯美化隆。睿范超千载，嘉猷备六宫。肃恭陪盛典，钦若荐禋宗。 132 | 三灵降飨，三后配神。虔敷藻奠，敬展郊禋。 133 | 已陈粢盛敷严祀，更奏笙镛协雅声。璇图宝历欣宁谧，晏俗淳风乐太平。 134 | 堂堂圣祖兴，赫赫昌基泰。戎车盟津偃，玉帛涂山会。舜日启祥晖，尧云卷征旆。风猷被有截，声教覃无外。 135 | 至矣丕构，烝哉太平。授牺膺箓，复禹继明。草木仁化，凫鹥颂声。祀宗陈德，无愧斯诚。 136 | 郊坛斋帝，礼乐祀天。丹青寰宇，宫征山川。神祇毕降，行止重旋。融融穆穆，纳祉洪延。 137 | 止奏潜聆，登仪宿啭。太玉躬奉，参钟首奠。簠簋聿升，牺牲递荐。昭事颙若，存存以伣。 138 | 烂云普洽，律风无外。千品其凝，九宾斯会。禋尊晋烛，纯牺涤汰。玄覆攸广，鸿休汪濊。 139 | 六变爰阕，八阶载虔。祐我皇祚，于万斯年。 140 | 於赫圣祖，龙飞晋阳。底定万国，奄有四方。功格上下，道冠农黄。郊天配享，德合无疆。 141 | 崇崇泰畤，肃肃严禋。粢盛既洁，金石毕陈。上帝来享，介福爰臻。受釐合祉，宝祚维新。 142 | 祝史正辞，人神叶庆。福以德招，享以诚应。六变云备，百礼期浃。祀事孔明。祚流万叶。 143 | 大号成命，思文配天。神光肸蚃，龙驾言旋。眇眇阊阖，昭昭上玄。俾昌而大。于斯万年。 144 | 六成既阕，三荐云终。神心具醉，圣敬愈崇。受釐皇邸，回跸帷宫。穰穰之福，永永无穷。 145 | 孝敬中发，和容外彰。腾华照宇，如升太阳。贞璧就奠，玄灵垂光。礼乐具举，济济洋洋。 146 | 奠祖配天，承天享帝。百灵咸秩，四海来祭。植我苍璧，布我玄制。华日裴回，神烟容裔。 147 | 俎豆有馥，粢盛洁丰。亦有和羹，既戒既平。钟鼓管磬，肃唱和鸣。皇皇后祖，来我思成。 148 | 烝烝我后，享献惟夤。躬酌郁鬯，跪奠明神。孝莫孝乎，配上帝亲。敬莫敬乎，孝天下臣。 149 | 皇祖严配，配享皇天。皇皇降嘏，天子万年。 150 | 六钟翕协六变成，八佾倘佯八风生。乐九韶兮人神感，美七德兮天地清。 151 | 烈祖顺三灵，文宗威四海。黄钺诛群盗，朱旗扫多罪。戢兵天下安，约法人心改。大哉干羽意，长见风云在。 152 | 乐已终，烟燎上。怀灵惠，结皇想。归风疾，回风爽。百福来，众神往。 153 | 履艮斯绳，居中体正。龙运垂祉，昭符启圣。式事严禋，聿怀嘉庆。惟帝永锡，时皇休命。 154 | 殷荐乘春，太坛临曙。八簋盈和，六瑚登御。嘉稷匪歆，德馨斯饫。祝嘏无易，灵心有豫。 155 | 玉帛牺牲申敬享，金丝戚羽盛音容。庶俾亿龄禔景福，长欣万宇洽时邕。 156 | 象天御宇，乘时布政。严配申虔，宗禋展敬。樽罍盈列，树羽交映。玉币通诚，祚隆皇圣。 157 | 八牖晨披，五精朝奠。雾凝璇篚，风清金县。神涤备全，明粢丰衍。载絜彝俎，陈诚以荐。 158 | 御扆合宫承宝历，席图重馆奉明灵。偃武修文九围泰，沉烽静柝八荒宁。 159 | 总章陈昔典，衢室礼惟神。宏规则天地，神用叶陶钧。负扆三春旦，充庭万宇宾。顾己诚虚薄，空惭亿兆人。 160 | 仰膺历数，俯顺讴歌。远安迩肃，俗阜时和。化光玉镜，讼息金科。方兴典礼，永戢干戈。 161 | 至人光俗，大孝通神。谦以表性，恭惟立身。洪规载启，茂典方陈。誉隆三善，祥开万春。 162 | 千官肃事，万国朝宗。载延百辟，爰集三宫。君臣德合，鱼水斯同。睿图方永，周历长隆。 163 | 礼崇宗祀，志表严禋。笙镛合奏，文物维新。敬遵茂典，敢择良辰。絜诚斯著，奠谒方申。 164 | 履艮包群望，居中冠百灵。万方资广运，庶品荷财成。神功谅匪测，盛德实难名。藻奠申诚敬，恭祀表惟馨。 165 | 出震位，开平秩。扇条风，乘甲乙。龙德盛，鸟星出。荐珪篚，陈诚实。 166 | 赫赫离精御炎陆，滔滔炽景开隆暑。冀延神鉴俯兰樽，式表虔襟陈桂俎。 167 | 律中夷则，序应收成。功宣建武，义表惟明。爰申礼奠，庶展翘诚。九秋是式，百谷斯盈。 168 | 葭律肇启隆冬，蘋藻攸陈飨祭。黄钟既陈玉烛，红粒方殷稔岁。 169 | 朱鸟开辰，苍龙启映。大帝昭飨，群生展敬。礼备怀柔，功宣舞咏。旬液应序，年祥协庆。 170 | 绀筵分彩，宝图吐绚。风管晨凝，云歌晓啭。肃事兰羞，虔申桂奠。百谷斯登，万箱攸荐。 171 | 凤曲登歌调令序，龙雩集舞泛祥风。彩旞云回昭睿德，朱干电发表神功。 172 | 鸟纬迁序，龙星见辰。纯阳在律，明德崇禋。五方降帝，万宇安人。恭以致享，肃以迎神。 173 | 祀遵经设，享缘成举。献毕于樽，彻临于俎。舞止干戚，乐停枧敔。歌以送神，神还其所。 174 | 万里扈封峦，群公遇此欢。幔城连夜静，霜仗满空寒。辇路宵烟合，旌门晓月残。明朝陪圣主，山下礼圆坛。 175 | 玉女贵妃生，嫛婗始发声。金盆浴未了，绷子绣初成。翡翠雕芳缛，真珠帖小缨。何时学健步，斗取落花轻。 176 | 社金流茂祉，庭玉表奇才。竹似因谈植，兰疑入梦栽。乌将八子去，凤逐九雏来。今夜明珠色，当随满月开。 177 | 王家傍绿池，春色正相宜。岂有楼台好，兼看草树奇。石榴天上叶，椰子日南枝。出入千门里，年年乐未移。 178 | 半额画双蛾，盈盈烛下歌。玉杯寒意少，金屋夜情多。香艳王分帖，裙娇敕赐罗。平阳莫相妒，唤出不如他。 179 | 朱绂临秦望，皇华赴洛桥。文章南渡越，书奏北归朝。树入江云尽，城衔海月遥。秋风将客思，川上晚萧萧。 180 | 天子爱贤才，星郎入拜来。明光朝半下，建礼直初回。名带含香发，文随绮幕开。披云自有镜，从此照仙台。 181 | 辍史空三署，题舆佐一方。祖筵开霁景，征陌直朝光。水陆风烟隔，秦吴道路长。伫闻敷善政，邦国咏惟康。 182 | 秋来林下不知春，一种佳游事也均。绛叶从朝飞著夜，黄花开日未成旬。将曛陌树频惊鸟，半醉归途数问人。城远登高并九日，茱萸凡作几年新。 183 | 花源药屿凤城西，翠幕纱窗莺乱啼。昨夜蒲萄初上架，今朝杨柳半垂堤。片片仙云来渡水，双双燕子共衔泥。请语东风催后骑，并将歌舞向前谿。 184 | 秋叶风吹黄飒飒，晴云日照白鳞鳞。归来得问茱萸女，今日登高醉几人。 185 | 朔风吹寒塞，胡沙千万里。陈云出岱山，孤月生海水。决胜方求敌，衔恩本轻死。萧萧牧马鸣，中夜拔剑起。 186 | 物情良可见，人事不胜悲。莫恃朝荣好，君看暮落时。 187 | 铜台宫观委灰尘，魏主园林漳水滨。即今西望犹堪思，况复当时歌舞人。 188 | 白锦文章乱，丹霄羽翮齐。云中呼暂下，雪里放还迷。梁苑惊池鹜，陈仓拂野鸡。不知寥廓外，何处独依栖。 189 | 岸与恩同广，波将慈共深。涓涓劳日夜，长似下流心。 190 | 於穆浚哲，维清缉熙。肃事昭配，永言孝思。涤濯静嘉，馨香在兹。神之听之，用受福釐。 191 | 红树晓莺啼，春风暖翠闺。雕笼熏绣被，珠履踏金堤。芍药花初吐，菖蒲叶正齐。藁砧当此日，行役向辽西。 192 | 主人病且闲，客来情弥适。一酌复一笑，不知日将夕。昨来属欢游，于今尽成昔。努力持所趣，空名定何益。 193 | 坤元载物，阳乐发生。播植资始，品汇咸亨。列俎棋布，方坛砥平。神歆禋祀，后德惟明。 194 | 早知君爱歇，本自无萦妒。谁使恩情深，今来反相误。愁眠罗帐晓，泣坐金闺暮。独有梦中魂，犹言意如故。 195 | 君爱本相饶，从来似舞腰。那堪攀玉座，肠断望陵朝。怨著情无主，哀凝曲不调。况临松日暮，悲吹坐萧萧。 196 | 正月金闺里，微风绣户间。晓魂怜别梦，春思逼啼颜。绕砌梅堪折，当轩树未攀。岁华庭北上，何日度阳关。 197 | 二月韶光好，春风香气多。园中花巧笑，林里鸟能歌。有恨离琴瑟，无情著绮罗。更听春燕语，妾亦不如他。 198 | 三月春将尽，空房妾独居。蛾眉愁自结，鬓发没情梳。□□□□□，□□□□□。□□□□□，□□□□□。 199 | 七月坐凉宵，金波满丽谯。容华芳意改，枕席怨情饶。锦字沾愁泪，罗裙缓细腰。不如银汉女，岁岁鹊成桥。 200 | 令节颁龙镜，仙辉下凤台。含灵万象入，写照百花开。色与皇明散，光随圣泽来。妍媸冰鉴里，从此愧非才。 201 | 整驾升车望寥廓，垂阴荐祉荡昏氛。飨时灵贶僾如在。乐罢余声遥可闻。饮福陈诚礼容备，撤俎终献曙光分。跪拜临坛结空想，年年应节候油云。 202 | 征戍动经年，含情拂玳筵。花飞织锦处，月落捣衣边。灯暗愁孤坐，床空怨独眠。自君辽海去，玉匣闭春弦。 203 | 楼前百戏竞争新，唯有长竿妙入神。谁谓绮罗翻有力，犹自嫌轻更著人。 204 | 汉祚惟永，神功中兴。夙驱氛祲，天覆黎蒸。三光再朗，庶绩其凝。重熙累叶，景命是膺。 205 | 寒仗丹旄引，阴堂白日违。暗灯明象物，画水湿灵衣。羽化淮王去，仙迎太子归。空馀燕衔士，朝夕向陵飞。 206 | 温室欢初就，兰交托胜因。共听无漏法，兼濯有为尘。水洁三空性，香沾四大身。清心多善友，颂德慰同人。 207 | 天门街里倒天枢，火急先须御火珠。计合一条丝线挽，何劳两县索人夫。 208 | 挂冠知止足，岂独汉疏贤。入道求真侣，辞恩访列仙。睿文含日月，宸翰动云烟。鹤驾吴乡远，遥遥南斗边。 209 | 东幸从人望，西巡顺物回。云收二华出，天转五星来。十月农初罢，三驱礼复开。更看琼岳上，佳气接神台。 210 | 一去一万里，千知千不还。崖州何处在，生度鬼门关。 211 | 雪面淡眉天上女，凤箫鸾翅欲飞去。玉山翘翠步无尘，楚腰如柳不胜春。 212 | 年来谁不厌龙钟，虽在侯门似不容。看取海山寒翠树，苦遭霜霰到秦封。 213 | 一片苍梧意，氤氲生栋梁。下帘山足暗，开户日添光。偏使衣裘润，能令枕簟凉。无心伴行雨，何必梦荆王。 214 | 啾啾青雀儿，飞来飞去仰天池。逍遥饮啄安涯分，何假扶摇九万为。 215 | 经书满腹中，吾识广川翁。年老甘无位，家贫懒发蒙。人归洙泗学，歌盛舞雩风。愿接诸生礼，三年事马融。 216 | 高唐几百里，树色接阳台。晚见江山霁，宵闻风雨来。云从三峡起，天向数峰开。灵境信难见，轻舟那可回。 217 | 江南冰不闭，山泽气潜通。腊月闻山鸟，寒崖见蛰熊。柳林春半合，荻笋乱无丛。回首金陵岸，依依向北风。 218 | 长风起秋色，细雨含落晖。夕鸟向林去，晚帆相逐飞。虫声出乱草，水气薄行衣。一别故乡道，悠悠今始归。 219 | 中禁夕沉沉，幽篁别作林。色连鸡树近，影落凤池深。为重凌霜节，能虚应物心。年年承雨露，长对紫庭阴。 220 | 传闻圣主幸新丰，清跸鸣銮出禁中。细草终朝随步辇，垂杨几处绕行宫。千官扈从骊山北，万国来朝渭水东。此日小臣徒献赋，汉家谁复重扬雄。 221 | 少妇石榴裙，新妆白玉面。能迷张公子，不许时相见。 222 | 映竹时闻转辘轳，当窗只见网蜘蛛。主人非病常高卧，环堵蒙笼一老儒。 223 | 与君相识即相亲，闻道君家住孟津。为见行舟试借问，客中时有洛阳人。 224 | 我年一何长，鬓发日已白。俯仰天地间，能为几时客。惆怅故山云，裴回空日夕。何事与时人，东城复南陌。 225 | 邑有弦歌宰，翔鸾狎野鸥。眷言华省旧，暂滞海池游。郁岛藏深竹，前谿对舞楼。更闻书即事，云物是新秋。 226 | 校文常近日，赐宴忽升天。酒正传杯至，饔人捧案前。玉阶鸣溜水，清阁引归烟。共惜芸香暮，春风几万年。 227 | 神女调温液，年年待圣人。试开临水殿，来洗属车尘。暖气随明主，恩波浃近臣。灵威自无极，从此献千春。 228 | 圣作西山颂，君其出使年。勒碑悬日月，驱传接云烟。寒尽函关路，春归洛水边。别离能几许，朝暮玉墀前。 229 | 郡县分南国，皇华出圣朝。为怜乡棹近，不道使车遥。旧俗吴三让，遗风汉六条。愿言除疾苦，天子听歌谣。 230 | 明月开三峡，花源出五溪。城池青壁里，烟火绿林西。不畏王程促，惟愁仙路迷。巴东下归棹，莫待夜猿啼。 231 | 画得襄阳郡，依然见昔游。岘山思驻马，汉水忆回舟。丹壑常含霁，青林不换秋。图书空咫尺，千里意悠悠。 232 | 琴瑟调双凤，和鸣不独飞。正歌春可乐，行泣露先晞。环珮声犹在，房栊梦不归。将军休沐日，谁劝著新衣。 233 | 两宫斋祭近登临，雨雪纷纷天昼阴。只为经寒无瑞色，顿教正月满春林。蓬莱北上旌门暗，花萼南归马迹深。自有三农歌帝力，还将万庾答尧心。 234 | 北斗横天夜欲阑，愁人倚月思无端。忽闻画阁秦筝逸，知是邻家赵女弹。曲成虚忆青蛾敛，调急遥怜玉指寒。银锁重关听未辟，不如眠去梦中看。 235 | 猿鸣三峡里，行客旧沾裳。复道从兹去，思君不暂忘。开襟春叶短，分手夏条长。独有幽庭桂，年年空自芳。 236 | 山径入修篁，深林蔽日光。夏云生嶂远，瀑水引溪长。秀迹逢皆胜，清芬坐转凉。回看玉樽夕，归路赏前忘。 237 | 新秋夜何爽，露下风转凄。一磬竹林外，千灯花塔西。 238 | 窗灯林霭里，闻磬水声中。更与龙华会，炉烟满夕风。 239 | 挂冠知止足，岂独汉疏贤。入道求真侣，辞恩访列仙。睿文含日月，宸翰动云烟。鹤驾吴乡远，遥遥南斗边。 240 | 东幸从人望，西巡顺物回。云收二华出，天转五星来。十月农初罢，三驱礼复开。更看琼岳上，佳气接神台。 241 | 一去一万里，千知千不还。崖州何处在，生度鬼门关。 242 | 雪面淡眉天上女，凤箫鸾翅欲飞去。玉山翘翠步无尘，楚腰如柳不胜春。 243 | 年来谁不厌龙钟，虽在侯门似不容。看取海山寒翠树，苦遭霜霰到秦封。 244 | 一片苍梧意，氤氲生栋梁。下帘山足暗，开户日添光。偏使衣裘润，能令枕簟凉。无心伴行雨，何必梦荆王。 245 | 啾啾青雀儿，飞来飞去仰天池。逍遥饮啄安涯分，何假扶摇九万为。 246 | 经书满腹中，吾识广川翁。年老甘无位，家贫懒发蒙。人归洙泗学，歌盛舞雩风。愿接诸生礼，三年事马融。 247 | 高唐几百里，树色接阳台。晚见江山霁，宵闻风雨来。云从三峡起，天向数峰开。灵境信难见，轻舟那可回。 248 | 江南冰不闭，山泽气潜通。腊月闻山鸟，寒崖见蛰熊。柳林春半合，荻笋乱无丛。回首金陵岸，依依向北风。 249 | 长风起秋色，细雨含落晖。夕鸟向林去，晚帆相逐飞。虫声出乱草，水气薄行衣。一别故乡道，悠悠今始归。 250 | 中禁夕沉沉，幽篁别作林。色连鸡树近，影落凤池深。为重凌霜节，能虚应物心。年年承雨露，长对紫庭阴。 251 | 传闻圣主幸新丰，清跸鸣銮出禁中。细草终朝随步辇，垂杨几处绕行宫。千官扈从骊山北，万国来朝渭水东。此日小臣徒献赋，汉家谁复重扬雄。 252 | 少妇石榴裙，新妆白玉面。能迷张公子，不许时相见。 253 | 映竹时闻转辘轳，当窗只见网蜘蛛。主人非病常高卧，环堵蒙笼一老儒。 254 | 与君相识即相亲，闻道君家住孟津。为见行舟试借问，客中时有洛阳人。 255 | 我年一何长，鬓发日已白。俯仰天地间，能为几时客。惆怅故山云，裴回空日夕。何事与时人，东城复南陌。 256 | 邑有弦歌宰，翔鸾狎野鸥。眷言华省旧，暂滞海池游。郁岛藏深竹，前谿对舞楼。更闻书即事，云物是新秋。 257 | 吴越山多秀，新安江甚清。（见《河岳英灵集》）书名会粹才偏逸，酒号屠苏味更醇。（《赠郑虔》，见《唐语林》）初疑轻烟淡古松，又似山开万仞峰。（《赠怀素》，见《颜真卿序》） 258 | 校文常近日，赐宴忽升天。酒正传杯至，饔人捧案前。玉阶鸣溜水，清阁引归烟。共惜芸香暮，春风几万年。 259 | 神女调温液，年年待圣人。试开临水殿，来洗属车尘。暖气随明主，恩波浃近臣。灵威自无极，从此献千春。 260 | 圣作西山颂，君其出使年。勒碑悬日月，驱传接云烟。寒尽函关路，春归洛水边。别离能几许，朝暮玉墀前。 261 | 郡县分南国，皇华出圣朝。为怜乡棹近，不道使车遥。旧俗吴三让，遗风汉六条。愿言除疾苦，天子听歌谣。 262 | 明月开三峡，花源出五溪。城池青壁里，烟火绿林西。不畏王程促，惟愁仙路迷。巴东下归棹，莫待夜猿啼。 263 | 画得襄阳郡，依然见昔游。岘山思驻马，汉水忆回舟。丹壑常含霁，青林不换秋。图书空咫尺，千里意悠悠。 264 | 琴瑟调双凤，和鸣不独飞。正歌春可乐，行泣露先晞。环珮声犹在，房栊梦不归。将军休沐日，谁劝著新衣。 265 | 两宫斋祭近登临，雨雪纷纷天昼阴。只为经寒无瑞色，顿教正月满春林。蓬莱北上旌门暗，花萼南归马迹深。自有三农歌帝力，还将万庾答尧心。 266 | 北斗横天夜欲阑，愁人倚月思无端。忽闻画阁秦筝逸，知是邻家赵女弹。曲成虚忆青蛾敛，调急遥怜玉指寒。银锁重关听未辟，不如眠去梦中看。 267 | 猿鸣三峡里，行客旧沾裳。复道从兹去，思君不暂忘。开襟春叶短，分手夏条长。独有幽庭桂，年年空自芳。 268 | 山径入修篁，深林蔽日光。夏云生嶂远，瀑水引溪长。秀迹逢皆胜，清芬坐转凉。回看玉樽夕，归路赏前忘。 269 | 新秋夜何爽，露下风转凄。一磬竹林外，千灯花塔西。 270 | 窗灯林霭里，闻磬水声中。更与龙华会，炉烟满夕风。 271 | 长安年少惜春残，争认慈恩紫牡丹。别有玉盘乘露冷，无人起就月中看。 272 | 莫将铅粉匣，不用镜花光。一去边城路，何情更画妆。影销胡地月，衣尽汉宫香。妾死非关命，都缘怨断肠。 273 | 匡庐旧业是谁主，吴越新居安此生。白发数茎归未得，青山一望计还成。鸦翻枫叶夕阳动，鹭立芦花秋水明。从此舍舟何所诣，酒旗歌扇正相迎。 274 | 吹角动行人，喧喧行人起。笳悲马嘶乱，争渡金河水。日暮沙漠陲，战声烟尘里。尽系名王颈，归来献天子。 275 | 皎洁明星高，苍茫远天曙。槐雾暗不开，城鸦鸣稍去。始闻高阁声，莫辨更衣处。银烛已成行，金门俨驺驭。 276 | 篱间犬迎吠，出屋候荆扉。岁晏输井税，山村人夜归。晚田始家食，馀布成我衣。讵肯无公事，烦君问是非。 277 | 万方资以化，交泰属升平。易从业惟简，得一道斯宁。具仪光玉帛，送舞变咸英。黍稷良非贵，明德信惟馨。 278 | 校文常近日，赐宴忽升天。酒正传杯至，饔人捧案前。玉阶鸣溜水，清阁引归烟。共惜芸香暮，春风几万年。 279 | 神女调温液，年年待圣人。试开临水殿，来洗属车尘。暖气随明主，恩波浃近臣。灵威自无极，从此献千春。 280 | 圣作西山颂，君其出使年。勒碑悬日月，驱传接云烟。寒尽函关路，春归洛水边。别离能几许，朝暮玉墀前。 281 | 郡县分南国，皇华出圣朝。为怜乡棹近，不道使车遥。旧俗吴三让，遗风汉六条。愿言除疾苦，天子听歌谣。 282 | 明月开三峡，花源出五溪。城池青壁里，烟火绿林西。不畏王程促，惟愁仙路迷。巴东下归棹，莫待夜猿啼。 283 | 画得襄阳郡，依然见昔游。岘山思驻马，汉水忆回舟。丹壑常含霁，青林不换秋。图书空咫尺，千里意悠悠。 284 | 琴瑟调双凤，和鸣不独飞。正歌春可乐，行泣露先晞。环珮声犹在，房栊梦不归。将军休沐日，谁劝著新衣。 285 | 两宫斋祭近登临，雨雪纷纷天昼阴。只为经寒无瑞色，顿教正月满春林。蓬莱北上旌门暗，花萼南归马迹深。自有三农歌帝力，还将万庾答尧心。 286 | 北斗横天夜欲阑，愁人倚月思无端。忽闻画阁秦筝逸，知是邻家赵女弹。曲成虚忆青蛾敛，调急遥怜玉指寒。银锁重关听未辟，不如眠去梦中看。 287 | 将军出紫塞，冒顿在乌贪。笳喧雁门北，阵翼龙城南。雕弓夜宛转，铁骑晓参潭。应须驻白日，为待战方酣。 288 | 城南征战多，城北无饥鸦。白骨马蹄下，谁言皆有家。城前水声苦，倏忽流万古。莫争城外地，城里有闲土。 289 | 巫山凌太清，岧峣类削成。霏霏暮雨合，霭霭朝云生。危峰入鸟道，深谷泻猿声。别有幽栖客，淹留攀桂情。 290 | 巫山望不极，望望下朝雰。莫辨啼猿树，徒看神女云。惊涛乱水脉，骤雨暗峰文。沾裳即此地，况复远思君。 291 | 巫山高不极，沓沓状奇新。暗谷疑风雨，幽岩若鬼神。月明三峡曙，潮满二江春。为问阳台夕，应知入梦人。 292 | 楚国巫山秀，清猿日夜啼。万重春树合，十二碧峰齐。峡出朝云下，江来暮雨西。阳台归路直，不畏向家迷。 293 | 巫峡见巴东，迢迢半出空。云藏神女馆，雨到楚王宫。朝暮泉声落，寒暄树色同。清猿不可听，偏在九秋中。 294 | 巫山十二峰，皆在碧虚中。回合云藏日，霏微雨带风。猿声寒过水，树色暮连空。愁向高唐望，清秋见楚宫。 295 | 何地早芳菲，宛在长门殿。夭桃色若绶，秾李光如练。啼鸟弄花疏，游蜂饮香遍。叹息春风起，飘零君不见。 296 | 芳树本多奇，年华复在斯。结翠成新幄，开红满旧枝。风归花历乱，日度影参差。容色朝朝落，思君君不知。 297 | 玉花珍簟上，金缕画屏开。晓月怜筝柱，春风忆镜台。筝柱春风吹晓月，芳树落花朝暝歇。稿砧刀头未有时，攀条拭泪坐相思。 298 | -------------------------------------------------------------------------------- /inspect_checkpoint.lua: -------------------------------------------------------------------------------- 1 | -- simple script that loads a checkpoint and prints its opts 2 | 3 | require 'torch' 4 | require 'nn' 5 | require 'nngraph' 6 | 7 | require 'util.OneHot' 8 | require 'util.misc' 9 | 10 | cmd = torch.CmdLine() 11 | cmd:text() 12 | cmd:text('Load a checkpoint and print its options and validation losses.') 13 | cmd:text() 14 | cmd:text('Options') 15 | cmd:argument('-model','model to load') 16 | cmd:option('-gpuid',0,'gpu to use') 17 | cmd:text() 18 | 19 | -- parse input params 20 | opt = cmd:parse(arg) 21 | 22 | if opt.gpuid >= 0 then 23 | print('using CUDA on GPU ' .. opt.gpuid .. '...') 24 | require 'cutorch' 25 | require 'cunn' 26 | cutorch.setDevice(opt.gpuid + 1) 27 | end 28 | 29 | local model = torch.load(opt.model) 30 | 31 | print('opt:') 32 | print(model.opt) 33 | print('val losses:') 34 | print(model.val_losses) 35 | 36 | -------------------------------------------------------------------------------- /lstm_rnn.lua: -------------------------------------------------------------------------------- 1 | 2 | --[[ 3 | 4 | This file trains a character-level multi-layer RNN on text data 5 | 6 | Code is based on implementation in 7 | https://github.com/oxford-cs-ml-2015/practical6 8 | but modified to have multi-layer support, GPU support, as well as 9 | many other common model/optimization bells and whistles. 10 | The practical6 code is in turn based on 11 | https://github.com/wojciechz/learning_to_execute 12 | which is turn based on other stuff in Torch, etc... (long lineage) 13 | 14 | ]]-- 15 | 16 | require 'torch' 17 | require 'nn' 18 | require 'nngraph' 19 | require 'optim' 20 | require 'lfs' 21 | 22 | require 'util.OneHot' 23 | require 'util.misc' 24 | local DataLoader = require 'util.PaddedLineSplitLoader' 25 | local model_utils = require 'util.model_utils' 26 | local LSTM = require 'model.LSTM' 27 | 28 | cmd = torch.CmdLine() 29 | cmd:text() 30 | cmd:text('Train a character-level language model') 31 | cmd:text() 32 | cmd:text('Options') 33 | -- data 34 | cmd:option('-data_dir','data/tinyshakespeare','data directory. Should contain the file input.txt with input data') 35 | -- model params 36 | cmd:option('-rnn_size', 128, 'size of LSTM internal state') 37 | cmd:option('-num_layers', 2, 'number of layers in the LSTM') 38 | cmd:option('-model', 'lstm', 'for now only lstm is supported. keep fixed') 39 | -- optimization 40 | cmd:option('-learning_rate',2e-3,'learning rate') 41 | cmd:option('-learning_rate_decay',0.97,'learning rate decay') 42 | cmd:option('-learning_rate_decay_after',10,'in number of epochs, when to start decaying the learning rate') 43 | cmd:option('-decay_rate',0.95,'decay rate for rmsprop') 44 | cmd:option('-dropout',0,'dropout for regularization, used after each RNN hidden layer. 0 = no dropout') 45 | cmd:option('-seq_length',50,'number of timesteps to unroll for') 46 | cmd:option('-batch_size',50,'number of sequences to train on in parallel') 47 | cmd:option('-max_epochs',50,'number of full passes through the training data') 48 | cmd:option('-grad_clip',5,'clip gradients at this value') 49 | cmd:option('-train_frac',0.95,'fraction of data that goes into train set') 50 | cmd:option('-val_frac',0.05,'fraction of data that goes into validation set') 51 | -- test_frac will be computed as (1 - train_frac - val_frac) 52 | cmd:option('-init_from', '', 'initialize network parameters from checkpoint at this path') 53 | -- bookkeeping 54 | cmd:option('-seed',123,'torch manual random number generator seed') 55 | cmd:option('-print_every',1,'how many steps/minibatches between printing out the loss') 56 | cmd:option('-eval_val_every',1000,'every how many iterations should we evaluate on validation data?') 57 | cmd:option('-checkpoint_dir', 'cv', 'output directory where checkpoints get written') 58 | cmd:option('-compress', 1, 'compress data output') 59 | cmd:option('-savefile','lstm','filename to autosave the checkpont to. Will be inside checkpoint_dir/') 60 | cmd:option('-savemodel',1,'save me') 61 | -- GPU/CPU 62 | cmd:option('-gpuid',0,'which gpu to use. -1 = use CPU') 63 | cmd:text() 64 | 65 | -- parse input params 66 | opt = cmd:parse(arg) 67 | torch.manualSeed(opt.seed) 68 | -- train / val / test split for data, in fractions 69 | local test_frac = math.max(0, 1 - (opt.train_frac + opt.val_frac)) 70 | local split_sizes = {opt.train_frac, opt.val_frac, test_frac} 71 | 72 | -- initialize cunn/cutorch for training on the GPU and fall back to CPU gracefully 73 | if opt.gpuid >= 0 then 74 | local ok, cunn = pcall(require, 'cunn') 75 | local ok2, cutorch = pcall(require, 'cutorch') 76 | if not ok then print('package cunn not found!') end 77 | if not ok2 then print('package cutorch not found!') end 78 | if ok and ok2 then 79 | print('using CUDA on GPU ' .. opt.gpuid .. '...') 80 | cutorch.setDevice(opt.gpuid + 1) -- note +1 to make it 0 indexed! sigh lua 81 | cutorch.manualSeed(opt.seed) 82 | else 83 | print('If cutorch and cunn are installed, your CUDA toolkit may be improperly configured.') 84 | print('Check your CUDA toolkit installation, rebuild cutorch and cunn, and try again.') 85 | print('Falling back on CPU mode') 86 | opt.gpuid = -1 -- overwrite user setting 87 | end 88 | end 89 | 90 | -- create the data loader class 91 | local loader = DataLoader.create(opt.data_dir, opt.batch_size, opt.seq_length, split_sizes) 92 | local vocab_size = loader.vocab_size -- the number of distinct characters 93 | local vocab = loader.vocab_mapping 94 | print('vocab size: ' .. vocab_size) 95 | local ivocab = {} 96 | for c,i in pairs(vocab) do ivocab[i] = c end 97 | -- make sure output directory exists 98 | if not path.exists(opt.checkpoint_dir) then lfs.mkdir(opt.checkpoint_dir) end 99 | 100 | -- define the model: prototypes for one timestep, then clone them in time 101 | local do_random_init = true 102 | if string.len(opt.init_from) > 0 then 103 | print('loading an LSTM from checkpoint ' .. opt.init_from) 104 | local checkpoint = torch.load(opt.init_from) 105 | protos = checkpoint.protos 106 | -- make sure the vocabs are the same 107 | local vocab_compatible = true 108 | for c,i in pairs(checkpoint.vocab) do 109 | if not vocab[c] == i then 110 | vocab_compatible = false 111 | end 112 | end 113 | assert(vocab_compatible, 'error, the character vocabulary for this dataset and the one in the saved checkpoint are not the same. This is trouble.') 114 | -- overwrite model settings based on checkpoint to ensure compatibility 115 | print('overwriting rnn_size=' .. checkpoint.opt.rnn_size .. ', num_layers=' .. checkpoint.opt.num_layers .. ' based on the checkpoint.') 116 | opt.rnn_size = checkpoint.opt.rnn_size 117 | opt.num_layers = checkpoint.opt.num_layers 118 | do_random_init = false 119 | else 120 | print('creating an LSTM with ' .. opt.num_layers .. ' layers') 121 | protos = {} 122 | protos.rnn = LSTM.lstm(vocab_size, opt.rnn_size, opt.num_layers, opt.dropout) 123 | protos.criterion = nn.ClassNLLCriterion() 124 | end 125 | 126 | -- the initial state of the cell/hidden states 127 | init_state = {} 128 | for L=1,opt.num_layers do 129 | local h_init = torch.zeros(opt.batch_size, opt.rnn_size) 130 | if opt.gpuid >=0 then h_init = h_init:cuda() end 131 | table.insert(init_state, h_init:clone()) 132 | table.insert(init_state, h_init:clone()) 133 | end 134 | 135 | -- ship the model to the GPU if desired 136 | if opt.gpuid >= 0 then 137 | for k,v in pairs(protos) do v:cuda() end 138 | end 139 | 140 | -- put the above things into one flattened parameters tensor 141 | params, grad_params = model_utils.combine_all_parameters(protos.rnn) 142 | 143 | -- initialization 144 | if do_random_init then 145 | params:uniform(-0.08, 0.08) -- small numbers uniform 146 | end 147 | 148 | print('number of parameters in the model: ' .. params:nElement()) 149 | -- make a bunch of clones after flattening, as that reallocates memory 150 | clones = {} 151 | for name,proto in pairs(protos) do 152 | print('cloning ' .. name) 153 | clones[name] = model_utils.clone_many_times(proto, opt.seq_length, not proto.parameters) 154 | end 155 | 156 | -- evaluate the loss over an entire split 157 | -- safe to use the same rnn for all the forward passes 158 | function eval_split(split_index, max_batches) 159 | print('evaluating loss over split index ' .. split_index) 160 | local n = loader.split_sizes[split_index] 161 | if max_batches ~= nil then n = math.min(max_batches, n) end 162 | 163 | loader:reset_batch_pointer(split_index) -- move batch iteration pointer for this split to front 164 | local loss = 0 165 | local rnn_state = {[0] = init_state} 166 | local sampleSize = 0 167 | 168 | for i = 1,n do -- iterate over batches in the split 169 | -- fetch a batch 170 | local x, y = loader:next_batch(split_index) 171 | if opt.gpuid >= 0 then -- ship the input arrays to GPU 172 | -- have to convert to float because integers can't be cuda()'d 173 | x = x:float():cuda() 174 | y = y:float():cuda() 175 | end 176 | -- forward pass 177 | sampleSize = x:size(2) 178 | for t=1,sampleSize do 179 | clones.rnn[1]:evaluate() -- for dropout proper functioning 180 | local lst = clones.rnn[1]:forward{x[{{}, t}], unpack(rnn_state[t-1])} 181 | rnn_state[t] = {} 182 | for i=1,#init_state do table.insert(rnn_state[t], lst[i]) end 183 | prediction = lst[#lst] 184 | loss = loss + clones.criterion[1]:forward(prediction, y[{{}, t}]) 185 | end 186 | -- carry over lstm state 187 | --rnn_state[0] = rnn_state[#rnn_state] 188 | rnn_state = {[0] = init_state} 189 | end 190 | 191 | loss = loss / sampleSize / n 192 | return loss 193 | end 194 | 195 | function sample_sequence(protos, length) 196 | protos.rnn:evaluate() 197 | -- fill with uniform probabilities over characters (? hmm) 198 | -- gprint('missing seed text, using uniform probability over first character') 199 | -- gprint('--------------------------') 200 | print("evaluate some test sequence") 201 | -- local prediction = torch.ShortTensor(1, #ivocab):fill(1)/(#ivocab) 202 | -- this is for dummy 203 | -- local prev_char = torch.ShortTensor{vocab['F']} 204 | local prev_char = torch.ShortTensor{24} 205 | io.write(ivocab[prev_char[1]]) 206 | if opt.gpuid >= 0 then prediction = prediction:cuda() end 207 | -- use argmax 208 | local current_state = {} 209 | for L = 1,opt.num_layers do 210 | -- c and h for all layers 211 | local h_init = torch.zeros(1, opt.rnn_size) 212 | if opt.gpuid >= 0 then h_init = h_init:cuda() end 213 | table.insert(current_state, h_init:clone()) 214 | table.insert(current_state, h_init:clone()) 215 | end 216 | local state_size = #current_state 217 | for i=1, length do 218 | local lst = protos.rnn:forward{prev_char, unpack(current_state)} 219 | current_state = {} 220 | for i=1,state_size do table.insert(current_state, lst[i]) end 221 | prediction = lst[#lst] 222 | local _, prev_char_ = prediction:max(2) 223 | prev_char = prev_char_:resize(1):short() 224 | -- print(ivocab[prev_char[1]] .. ' prob ' .. prediction[1][prev_char[1]]) 225 | io.write(ivocab[prev_char[1]]) 226 | end 227 | end 228 | 229 | -- do fwd/bwd and return loss, grad_params 230 | local init_state_global = clone_list(init_state) 231 | function feval(x) 232 | if x ~= params then 233 | params:copy(x) 234 | end 235 | grad_params:zero() 236 | 237 | ------------------ get minibatch ------------------- 238 | local x, y = loader:next_batch(1) 239 | if opt.gpuid >= 0 then -- ship the input arrays to GPU 240 | -- have to convert to float because integers can't be cuda()'d 241 | x = x:float():cuda() 242 | y = y:float():cuda() 243 | end 244 | ------------------- forward pass ------------------- 245 | local loss = 0 246 | -- looks like stuff outside of seq_length didn't matter? 247 | -- both wojzaremba and karpathy didn't preserve drnn after propagated seq_length step 248 | local sampleSize = x:size(2) 249 | local trainingBatch = math.floor(sampleSize / opt.seq_length) 250 | for b=1, trainingBatch+1 do 251 | local rnn_state = {[0] = init_state_global} 252 | local predictions = {} -- softmax outputs 253 | local offset = (b-1) * opt.seq_length 254 | if b == trainingBatch+1 then 255 | offEnd = sampleSize - trainingBatch * opt.seq_length 256 | else 257 | offEnd = opt.seq_length 258 | end 259 | for t=1,offEnd do 260 | local dataPos = offset + t 261 | clones.rnn[t]:training() -- make sure we are in correct mode (this is cheap, sets flag) 262 | local lst = clones.rnn[t]:forward{x[{{}, dataPos}], unpack(rnn_state[t-1])} 263 | rnn_state[t] = {} 264 | for i=1,#init_state do table.insert(rnn_state[t], lst[i]) end -- extract the state, without output 265 | predictions[t] = lst[#lst] -- last element is the prediction 266 | loss = loss + clones.criterion[t]:forward(predictions[t], y[{{}, dataPos}]) 267 | end 268 | ------------------ backward pass ------------------- 269 | -- initialize gradient at time t to be zeros (there's no influence from future) 270 | local drnn_state = {[offEnd] = clone_list(init_state, true)} -- true also zeros the clones 271 | for t=offEnd,1,-1 do 272 | local dataPos = offset + t 273 | -- backprop through loss, and softmax/linear 274 | local doutput_t = clones.criterion[t]:backward(predictions[t], y[{{}, dataPos}]) 275 | table.insert(drnn_state[t], doutput_t) 276 | local dlst = clones.rnn[t]:backward({x[{{}, dataPos}], unpack(rnn_state[t-1])}, drnn_state[t]) 277 | drnn_state[t-1] = {} 278 | for k,v in pairs(dlst) do 279 | if k > 1 or b ~= 1 then -- k == 1 is gradient on x, which we dont need 280 | -- note we do k-1 because first item is dembeddings, and then follow the 281 | -- derivatives of the state, starting at index 2. I know... 282 | drnn_state[t-1][k-1] = v 283 | end 284 | end 285 | end 286 | ------------------------ misc ---------------------- 287 | -- transfer final state to initial state (BPTT) 288 | init_state_global = rnn_state[#rnn_state] -- NOTE: I don't think this needs to be a clone, right? 289 | end 290 | loss = loss / sampleSize 291 | -- clip gradient element-wise 292 | grad_params:clamp(-opt.grad_clip, opt.grad_clip) 293 | return loss, grad_params 294 | end 295 | 296 | -- start optimization here 297 | train_losses = {} 298 | val_losses = {} 299 | local val_loss = eval_split(2) -- 2 = validation 300 | val_losses[0] = val_loss 301 | print('initial validation loss is ' .. val_loss) 302 | local optim_state = {learningRate = opt.learning_rate, alpha = opt.decay_rate} 303 | local iterations = opt.max_epochs * loader.ntrain 304 | local iterations_per_epoch = loader.ntrain 305 | local loss0 = nil 306 | for i = 1, iterations do 307 | local epoch = i / loader.ntrain 308 | 309 | local timer = torch.Timer() 310 | local _, loss = optim.rmsprop(feval, params, optim_state) 311 | local time = timer:time().real 312 | 313 | local train_loss = loss[1] -- the loss is inside a list, pop it 314 | train_losses[i] = train_loss 315 | 316 | -- exponential learning rate decay 317 | if i % loader.ntrain == 0 and opt.learning_rate_decay < 1 then 318 | if epoch >= opt.learning_rate_decay_after then 319 | local decay_factor = opt.learning_rate_decay 320 | optim_state.learningRate = optim_state.learningRate * decay_factor -- decay it 321 | print('decayed learning rate by a factor ' .. decay_factor .. ' to ' .. optim_state.learningRate) 322 | end 323 | end 324 | 325 | -- every now and then or on last iteration 326 | if i % math.floor(opt.eval_val_every* iterations_per_epoch) == 0 or i == iterations then 327 | -- evaluate loss on validation data 328 | local val_loss = eval_split(2) -- 2 = validation 329 | val_losses[i] = val_loss 330 | print("validation loss at " .. i .. ' is ' .. val_loss) 331 | sample_sequence(protos, 10) 332 | 333 | if i ~= 1 and opt.savemodel == 1 then 334 | local savefile = string.format('%s/lm_%s_epoch%.2f_%.4f.t7', opt.checkpoint_dir, opt.savefile, epoch, val_loss) 335 | print('saving checkpoint to ' .. savefile) 336 | local checkpoint = {} 337 | checkpoint.protos = protos 338 | checkpoint.opt = opt 339 | checkpoint.train_losses = train_losses 340 | checkpoint.val_loss = val_loss 341 | checkpoint.val_losses = val_losses 342 | checkpoint.i = i 343 | checkpoint.epoch = epoch 344 | checkpoint.vocab = loader.vocab_mapping 345 | torch.save(savefile, checkpoint) 346 | if opt.compress == 1 then 347 | os.execute("pigz " .. savefile) 348 | end 349 | end 350 | end 351 | 352 | if i % opt.print_every == 0 then 353 | print(string.format("%d/%d (epoch %.3f), train_loss = %6.8f, grad/param norm = %6.4e, time/batch = %.2fs", i, iterations, epoch, train_loss, grad_params:norm() / params:norm(), time)) 354 | end 355 | 356 | if i % 10 == 0 then collectgarbage() end 357 | 358 | -- handle early stopping if things are going really bad 359 | if loss0 == nil then loss0 = loss[1] end 360 | if loss[1] > loss0 * 3 then 361 | print('loss is exploding, aborting.') 362 | break -- halt 363 | end 364 | end 365 | 366 | 367 | -------------------------------------------------------------------------------- /model/GRU.lua: -------------------------------------------------------------------------------- 1 | 2 | local GRU = {} 3 | 4 | --[[ 5 | Creates one timestep of one GRU 6 | Paper reference: http://arxiv.org/pdf/1412.3555v1.pdf 7 | ]]-- 8 | function GRU.gru(input_size, rnn_size, n) 9 | 10 | -- there are n+1 inputs (hiddens on each layer and x) 11 | local inputs = {} 12 | table.insert(inputs, nn.Identity()()) -- x 13 | for L = 1,n do 14 | table.insert(inputs, nn.Identity()()) -- prev_h[L] 15 | end 16 | 17 | function new_input_sum(insize, xv, hv) 18 | local i2h = nn.Linear(insize, rnn_size)(xv) 19 | local h2h = nn.Linear(rnn_size, rnn_size)(hv) 20 | return nn.CAddTable()({i2h, h2h}) 21 | end 22 | 23 | local x, input_size_L 24 | local outputs = {} 25 | for L = 1,n do 26 | 27 | local prev_h = inputs[L+1] 28 | if L == 1 then x = inputs[1] else x = outputs[L-1] end 29 | if L == 1 then input_size_L = input_size else input_size_L = rnn_size end 30 | 31 | -- GRU tick 32 | -- forward the update and reset gates 33 | local update_gate = nn.Sigmoid()(new_input_sum(input_size_L, x, prev_h)) 34 | local reset_gate = nn.Sigmoid()(new_input_sum(input_size_L, x, prev_h)) 35 | -- compute candidate hidden state 36 | local gated_hidden = nn.CMulTable()({reset_gate, prev_h}) 37 | local p2 = nn.Linear(rnn_size, rnn_size)(gated_hidden) 38 | local p1 = nn.Linear(input_size_L, rnn_size)(x) 39 | local hidden_candidate = nn.Tanh()(nn.CAddTable()({p1,p2})) 40 | -- compute new interpolated hidden state, based on the update gate 41 | local zh = nn.CMulTable()({update_gate, hidden_candidate}) 42 | local zhm1 = nn.CMulTable()({nn.AddConstant(1,false)(nn.MulConstant(-1,false)(update_gate)), prev_h}) 43 | local next_h = nn.CAddTable()({zh, zhm1}) 44 | 45 | table.insert(outputs, next_h) 46 | end 47 | 48 | return nn.gModule(inputs, outputs) 49 | end 50 | 51 | return GRU 52 | 53 | -------------------------------------------------------------------------------- /model/LSTM.lua: -------------------------------------------------------------------------------- 1 | 2 | local LSTM = {} 3 | function LSTM.lstm(input_size, rnn_size, n, dropout) 4 | dropout = dropout or 0 5 | 6 | -- there will be 2*n+1 inputs 7 | local inputs = {} 8 | table.insert(inputs, nn.Identity()()) -- x 9 | for L = 1,n do 10 | table.insert(inputs, nn.Identity()()) -- prev_c[L] 11 | table.insert(inputs, nn.Identity()()) -- prev_h[L] 12 | end 13 | 14 | local x, input_size_L 15 | local outputs = {} 16 | for L = 1,n do 17 | -- c,h from previos timesteps 18 | local prev_h = inputs[L*2+1] 19 | local prev_c = inputs[L*2] 20 | -- the input to this layer 21 | if L == 1 then 22 | x = OneHot(input_size)(inputs[1]) 23 | input_size_L = input_size 24 | else 25 | x = outputs[(L-1)*2] 26 | if dropout > 0 then x = nn.Dropout(dropout)(x) end -- apply dropout, if any 27 | input_size_L = rnn_size 28 | end 29 | -- evaluate the input sums at once for efficiency 30 | local i2h = nn.Linear(input_size_L, 4 * rnn_size)(x) 31 | local h2h = nn.Linear(rnn_size, 4 * rnn_size)(prev_h) 32 | local all_input_sums = nn.CAddTable()({i2h, h2h}) 33 | -- decode the gates 34 | local sigmoid_chunk = nn.Narrow(2, 1, 3 * rnn_size)(all_input_sums) 35 | sigmoid_chunk = nn.Sigmoid()(sigmoid_chunk) 36 | local in_gate = nn.Narrow(2, 1, rnn_size)(sigmoid_chunk) 37 | local forget_gate = nn.Narrow(2, rnn_size + 1, rnn_size)(sigmoid_chunk) 38 | local out_gate = nn.Narrow(2, 2 * rnn_size + 1, rnn_size)(sigmoid_chunk) 39 | -- decode the write inputs 40 | local in_transform = nn.Narrow(2, 3 * rnn_size + 1, rnn_size)(all_input_sums) 41 | in_transform = nn.Tanh()(in_transform) 42 | -- perform the LSTM update 43 | local next_c = nn.CAddTable()({ 44 | nn.CMulTable()({forget_gate, prev_c}), 45 | nn.CMulTable()({in_gate, in_transform}) 46 | }) 47 | -- gated cells form the output 48 | local next_h = nn.CMulTable()({out_gate, nn.Tanh()(next_c)}) 49 | 50 | table.insert(outputs, next_c) 51 | table.insert(outputs, next_h) 52 | end 53 | 54 | -- set up the decoder 55 | local top_h = outputs[#outputs] 56 | if dropout > 0 then top_h = nn.Dropout(dropout)(top_h) end 57 | local proj = nn.Linear(rnn_size, input_size)(top_h) 58 | local logsoft = nn.LogSoftMax()(proj) 59 | table.insert(outputs, logsoft) 60 | 61 | return nn.gModule(inputs, outputs) 62 | end 63 | 64 | return LSTM 65 | 66 | -------------------------------------------------------------------------------- /model/RNN.lua: -------------------------------------------------------------------------------- 1 | local RNN = {} 2 | 3 | function RNN.rnn(input_size, rnn_size, n, dropout) 4 | dropout = dropout or 0 5 | 6 | -- there are n+1 inputs (hiddens on each layer and x) 7 | local inputs = {} 8 | table.insert(inputs, nn.Identity()()) -- x 9 | for L = 1,n do 10 | table.insert(inputs, nn.Identity()()) -- prev_h[L] 11 | end 12 | 13 | -- do I really want to assume this code works? the prev_h and the x seems to be flipped? 14 | local x, input_size_L 15 | local outputs = {} 16 | for L = 1,n do 17 | local prev_h = inputs[L+1] 18 | if L == 1 then 19 | x = OneHot(input_size)(inputs[1]) 20 | input_size_L = input_size 21 | else 22 | x = outputs[L-1] 23 | if dropout > 0 then x = nn.Dropout(dropout)(x) end -- apply dropout, if any 24 | input_size_L = rnn_size 25 | end 26 | 27 | -- RNN tick 28 | local i2h = nn.Linear(input_size_L, rnn_size)(x) 29 | local h2h = nn.Linear(rnn_size, rnn_size)(prev_h) 30 | local next_h = nn.Tanh()(nn.CAddTable(){i2h, h2h}) 31 | 32 | table.insert(outputs, next_h) 33 | end 34 | 35 | -- set up the decoder 36 | local top_h = outputs[#outputs] 37 | if dropout > 0 then top_h = nn.Dropout(dropout)(top_h) end 38 | local proj = nn.Linear(rnn_size, input_size)(top_h) 39 | local logsoft = nn.LogSoftMax()(proj) 40 | table.insert(outputs, logsoft) 41 | 42 | return nn.gModule(inputs, outputs) 43 | end 44 | 45 | function nn.Recurrent:forgetEvalute() 46 | self.step = 1 47 | for i,_ in pairs(self.inputs) do 48 | self.inputs[i] = nil 49 | self.gradOutputs[i] = nil 50 | end 51 | for i, _ in pairs(self.gradOutputs) do 52 | self.gradOutputs[i] = nil 53 | end 54 | for i, _ in pairs(self.sharedClones) do 55 | self.sharedClones[i] = nil 56 | end 57 | local count=0 58 | local ta = {} 59 | --[[for i, _ in pairs(self.outputs) do 60 | table.insert(ta, i) 61 | self.outputs[i] = nil 62 | count = count+1 63 | end]]-- 64 | self.outputs = {} 65 | end 66 | 67 | function RNN.errnn(input_size, rnn_size,n,rho, dropout) 68 | -- TODO feedback function can contain dropout, namely nn.linear 69 | rho = rho or 8 70 | dropout = dropout or 0 71 | local hiddenSize = rnn_size 72 | local nIndex = input_size 73 | local feedback = nn.Linear(hiddenSize,hiddenSize) 74 | if dropout > 0 then 75 | local dfeed = nn.Sequential() 76 | dfeed:add(feedback) 77 | dfeed:add(nn.Dropout(dropout)) 78 | feedback = dfeed 79 | end 80 | --[[local r = nn.Recurrent( 81 | hiddenSize, nn.LookupTable(nIndex, hiddenSize), 82 | nn.Linear(hiddenSize, hiddenSize), nn.Sigmoid(), 83 | rho 84 | )]]-- 85 | local r = nn.Recurrent( 86 | hiddenSize, nn.LookupTable(nIndex, hiddenSize), 87 | feedback, nn.ReLU(), 88 | rho 89 | ) 90 | rnn = nn.Sequential() 91 | rnn:add(r) 92 | if dropout > 0 then 93 | rnn:add(nn.Dropout(dropout)) 94 | end 95 | rnn:add(nn.Linear(hiddenSize, nIndex)) 96 | rnn:add(nn.LogSoftMax()) 97 | return rnn,r 98 | end 99 | 100 | return RNN 101 | -------------------------------------------------------------------------------- /sample.lua: -------------------------------------------------------------------------------- 1 | 2 | --[[ 3 | 4 | This file samples characters from a trained model 5 | 6 | Code is based on implementation in 7 | https://github.com/oxford-cs-ml-2015/practical6 8 | 9 | ]]-- 10 | 11 | require 'torch' 12 | require 'nn' 13 | require 'nngraph' 14 | require 'optim' 15 | require 'lfs' 16 | 17 | require 'util.OneHot' 18 | require 'util.misc' 19 | 20 | cmd = torch.CmdLine() 21 | cmd:text() 22 | cmd:text('Sample from a character-level language model') 23 | cmd:text() 24 | cmd:text('Options') 25 | -- required: 26 | cmd:argument('-model','model checkpoint to use for sampling') 27 | -- optional parameters 28 | cmd:option('-seed',123,'random number generator\'s seed') 29 | cmd:option('-sample',1,' 0 to use max at each timestep, 1 to sample at each timestep') 30 | cmd:option('-primetext',"",'used as a prompt to "seed" the state of the LSTM using a given sequence, before we sample.') 31 | cmd:option('-length',2000,'number of characters to sample') 32 | cmd:option('-temperature',1,'temperature of sampling') 33 | cmd:option('-gpuid',0,'which gpu to use. -1 = use CPU') 34 | cmd:option('-verbose',1,'set to 0 to ONLY print the sampled text, no diagnostics') 35 | cmd:text() 36 | 37 | -- parse input params 38 | opt = cmd:parse(arg) 39 | 40 | -- gated print: simple utility function wrapping a print 41 | function gprint(str) 42 | if opt.verbose == 1 then print(str) end 43 | end 44 | 45 | -- check that cunn/cutorch are installed if user wants to use the GPU 46 | if opt.gpuid >= 0 then 47 | local ok, cunn = pcall(require, 'cunn') 48 | local ok2, cutorch = pcall(require, 'cutorch') 49 | if not ok then gprint('package cunn not found!') end 50 | if not ok2 then gprint('package cutorch not found!') end 51 | if ok and ok2 then 52 | gprint('using CUDA on GPU ' .. opt.gpuid .. '...') 53 | cutorch.setDevice(opt.gpuid + 1) -- note +1 to make it 0 indexed! sigh lua 54 | cutorch.manualSeed(opt.seed) 55 | else 56 | gprint('Falling back on CPU mode') 57 | opt.gpuid = -1 -- overwrite user setting 58 | end 59 | end 60 | torch.manualSeed(opt.seed) 61 | 62 | -- load the model checkpoint 63 | if not lfs.attributes(opt.model, 'mode') then 64 | gprint('Error: File ' .. opt.model .. ' does not exist. Are you sure you didn\'t forget to prepend cv/ ?') 65 | end 66 | checkpoint = torch.load(opt.model) 67 | protos = checkpoint.protos 68 | protos.rnn:evaluate() -- put in eval mode so that dropout works properly 69 | 70 | -- initialize the vocabulary (and its inverted version) 71 | local vocab = checkpoint.vocab 72 | local ivocab = {} 73 | for c,i in pairs(vocab) do ivocab[i] = c end 74 | 75 | -- initialize the rnn state to all zeros 76 | gprint('creating an LSTM...') 77 | local current_state 78 | local num_layers = checkpoint.opt.num_layers 79 | current_state = {} 80 | for L = 1,checkpoint.opt.num_layers do 81 | -- c and h for all layers 82 | local h_init = torch.zeros(1, checkpoint.opt.rnn_size) 83 | if opt.gpuid >= 0 then h_init = h_init:cuda() end 84 | table.insert(current_state, h_init:clone()) 85 | table.insert(current_state, h_init:clone()) 86 | end 87 | state_size = #current_state 88 | 89 | -- do a few seeded timesteps 90 | local seed_text = opt.primetext 91 | if string.len(seed_text) > 0 then 92 | gprint('seeding with ' .. seed_text) 93 | gprint('--------------------------') 94 | for c in string.gfind(seed_text, "([%z\1-\127\194-\244][\128-\191]*)") do 95 | prev_char = torch.Tensor{vocab[c]} 96 | io.write(ivocab[prev_char[1]]) 97 | if opt.gpuid >= 0 then prev_char = prev_char:cuda() end 98 | local lst = protos.rnn:forward{prev_char, unpack(current_state)} 99 | -- lst is a list of [state1,state2,..stateN,output]. We want everything but last piece 100 | current_state = {} 101 | for i=1,state_size do table.insert(current_state, lst[i]) end 102 | prediction = lst[#lst] -- last element holds the log probabilities 103 | end 104 | else 105 | -- fill with uniform probabilities over characters (? hmm) 106 | gprint('missing seed text, using uniform probability over first character') 107 | gprint('--------------------------') 108 | prediction = torch.Tensor(1, #ivocab):fill(1)/(#ivocab) 109 | if opt.gpuid >= 0 then prediction = prediction:cuda() end 110 | end 111 | 112 | -- start sampling/argmaxing 113 | for i=1, opt.length do 114 | 115 | -- log probabilities from the previous timestep 116 | if opt.sample == 0 then 117 | -- use argmax 118 | local _, prev_char_ = prediction:max(2) 119 | prev_char = prev_char_:resize(1) 120 | else 121 | -- use sampling 122 | prediction:div(opt.temperature) -- scale by temperature 123 | local probs = torch.exp(prediction):squeeze() 124 | probs:div(torch.sum(probs)) -- renormalize so probs sum to one 125 | prev_char = torch.multinomial(probs:float(), 1):resize(1):float() 126 | end 127 | 128 | -- forward the rnn for next character 129 | local lst = protos.rnn:forward{prev_char, unpack(current_state)} 130 | current_state = {} 131 | for i=1,state_size do table.insert(current_state, lst[i]) end 132 | prediction = lst[#lst] -- last element holds the log probabilities 133 | 134 | io.write(ivocab[prev_char[1]]) 135 | end 136 | io.write('\n') io.flush() 137 | 138 | -------------------------------------------------------------------------------- /sample_qts.lua: -------------------------------------------------------------------------------- 1 | 2 | --[[ 3 | th sample.lua -length 20 -primetext '月' -temperature 1 -verbose 0 cv/qts_jinti1/lm_jinti_25_epoch17.50_3.0796.t7 >> moon_sample.txt 4 | This file samples characters from a trained model 5 | 6 | Code is based on implementation in 7 | https://github.com/oxford-cs-ml-2015/practical6 8 | 9 | ]]-- 10 | 11 | require 'torch' 12 | require 'nn' 13 | require 'nngraph' 14 | require 'lfs' 15 | 16 | require 'util.OneHot' 17 | require 'util.misc' 18 | 19 | cmd = torch.CmdLine() 20 | cmd:text() 21 | cmd:text('Sample from a character-level language model') 22 | cmd:text() 23 | cmd:text('Options') 24 | -- required: 25 | cmd:argument('-model','model checkpoint to use for sampling') 26 | -- optional parameters 27 | cmd:option('-seed',123,'random number generator\'s seed') 28 | cmd:option('-sample',1,' 0 to use max at each timestep, 1 to sample at each timestep') 29 | cmd:option('-primetext',"",'used as a prompt to "seed" the state of the LSTM using a given sequence, before we sample.') 30 | cmd:option('-length',20,'number of characters to sample') 31 | cmd:option('-temperature',1,'temperature of sampling') 32 | cmd:option('-gpuid',0,'which gpu to use. -1 = use CPU') 33 | cmd:option('-verbose',1,'set to 0 to ONLY print the sampled text, no diagnostics') 34 | cmd:text() 35 | 36 | -- parse input params 37 | opt = cmd:parse(arg) 38 | 39 | -- gated print: simple utility function wrapping a print 40 | function gprint(str) 41 | if opt.verbose == 1 then print(str) end 42 | end 43 | 44 | -- check that cunn/cutorch are installed if user wants to use the GPU 45 | if opt.gpuid >= 0 then 46 | local ok, cunn = pcall(require, 'cunn') 47 | local ok2, cutorch = pcall(require, 'cutorch') 48 | if not ok then gprint('package cunn not found!') end 49 | if not ok2 then gprint('package cutorch not found!') end 50 | if ok and ok2 then 51 | gprint('using CUDA on GPU ' .. opt.gpuid .. '...') 52 | cutorch.setDevice(opt.gpuid + 1) -- note +1 to make it 0 indexed! sigh lua 53 | cutorch.manualSeed(opt.seed) 54 | else 55 | gprint('Falling back on CPU mode') 56 | opt.gpuid = -1 -- overwrite user setting 57 | end 58 | end 59 | torch.manualSeed(opt.seed) 60 | 61 | local Yunjiao = require 'yunjiao.YunjiaoLoader' 62 | local yunjiao = Yunjiao.create() 63 | 64 | -- these characters gets removed during sampling 65 | local exclusion = {'*', '□'} 66 | -- load the model checkpoint 67 | if not lfs.attributes(opt.model, 'mode') then 68 | gprint('Error: File ' .. opt.model .. ' does not exist. Are you sure you didn\'t forget to prepend cv/ ?') 69 | end 70 | checkpoint = torch.load(opt.model) 71 | protos = checkpoint.protos 72 | protos.rnn:evaluate() -- put in eval mode so that dropout works properly 73 | 74 | -- initialize the vocabulary (and its inverted version) 75 | local vocab = checkpoint.vocab 76 | local ivocab = {} 77 | for c,i in pairs(vocab) do ivocab[i] = c end 78 | 79 | -- initialize the rnn state to all zeros 80 | gprint('creating an LSTM with layer ...' .. checkpoint.opt.num_layers) 81 | local current_state 82 | local num_layers = checkpoint.opt.num_layers 83 | current_state = {} 84 | for L = 1,checkpoint.opt.num_layers do 85 | -- c and h for all layers 86 | local h_init = torch.zeros(1, checkpoint.opt.rnn_size) 87 | if opt.gpuid >= 0 then h_init = h_init:cuda() end 88 | table.insert(current_state, h_init:clone()) 89 | table.insert(current_state, h_init:clone()) 90 | end 91 | state_size = #current_state 92 | 93 | -- do a few seeded timesteps 94 | local seed_text = '^' .. opt.primetext 95 | for c in string.gfind(seed_text, "([%z\1-\127\194-\244][\128-\191]*)") do 96 | if vocab[c] == nil then 97 | print(c .. ' is not in the dictionary, replace with generic character') 98 | end 99 | end 100 | 101 | -- start sampling/argmaxing 102 | local topprint = 20 103 | local probs 104 | local poemCount = 0 105 | for k=1, opt.length*2 do 106 | -- max length for a poem is 64 107 | local furtherexclusion = {} 108 | local current_state = {} 109 | for L = 1,checkpoint.opt.num_layers do 110 | -- c and h for all layers 111 | local h_init = torch.zeros(1, checkpoint.opt.rnn_size) 112 | if opt.gpuid >= 0 then h_init = h_init:cuda() end 113 | table.insert(current_state, h_init:clone()) 114 | table.insert(current_state, h_init:clone()) 115 | end 116 | 117 | -- always seeding with ^ at least 118 | if string.len(seed_text) > 0 then 119 | gprint('seeding with ' .. seed_text) 120 | gprint('--------------------------') 121 | for c in string.gfind(seed_text, "([%z\1-\127\194-\244][\128-\191]*)") do 122 | if vocab[c] == nil then 123 | c = '*' 124 | end 125 | prev_char = torch.FloatTensor{vocab[c]} 126 | if opt.gpuid >= 0 then prev_char = prev_char:cuda() end 127 | local lst = protos.rnn:forward{prev_char, unpack(current_state)} 128 | -- lst is a list of [state1,state2,..stateN,output]. We want everything but last piece 129 | current_state = {} 130 | for i=1,state_size do table.insert(current_state, lst[i]) end 131 | prediction = lst[#lst] -- last element holds the log probabilities 132 | end 133 | else 134 | -- fill with uniform probabilities over characters (? hmm) 135 | gprint('missing seed text, using uniform probability over first character') 136 | gprint('--------------------------') 137 | prediction = torch.Tensor(1, #ivocab):fill(1)/(#ivocab) 138 | if opt.gpuid >= 0 then prediction = prediction:cuda() end 139 | end 140 | local poem = opt.primetext 141 | 142 | local looprunning = true 143 | for i=1, 64 do 144 | if not looprunning then 145 | break 146 | end 147 | -- log probabilities from the previous timestep 148 | local expectedPingze = yunjiao:getNextPingzes(poem) 149 | if #expectedPingze == 0 then 150 | error(poem .. ' is not possible to continue') 151 | end 152 | 153 | if opt.sample == 0 then 154 | -- use argmax 155 | local _, prev_char_ = prediction:max(2) 156 | prev_char = prev_char_:resize(1) 157 | probs = torch.exp(prediction):squeeze() 158 | probs:div(torch.sum(probs)) -- renormalize so probs sum to one 159 | else 160 | -- previous present characters 161 | for _, c in ipairs(furtherexclusion) do 162 | -- prediction[1][vocab[c]] = -100 163 | prediction[1][vocab[c]] = prediction[1][vocab[c]] * 2 164 | end 165 | -- forbidden characters 166 | for _, c in ipairs(exclusion) do 167 | prediction[1][vocab[c]] = -100 168 | end 169 | -- use sampling 170 | prediction:div(opt.temperature) -- scale by temperature 171 | probs = torch.exp(prediction):squeeze() 172 | probs:div(torch.sum(probs)) -- renormalize so probs sum to one 173 | 174 | for m = 1, 20 do 175 | prev_char = torch.multinomial(probs:float(), 1):resize(1):float() 176 | local tmp_char = ivocab[prev_char[1]] 177 | if tmp_char == '$' then 178 | break 179 | elseif yunjiao:belongsTo(expectedPingze, tmp_char ) then 180 | break 181 | else 182 | if opt.verbose == 1 then 183 | print(tmp_char .. " does not match pingze") 184 | end 185 | end 186 | if m == 20 then 187 | if opt.verbose == 1 then 188 | print(poem .. " does not fit pingze") 189 | end 190 | looprunning = false 191 | break 192 | end 193 | end 194 | end 195 | 196 | -- already picked, good to go 197 | local real_char = ivocab[prev_char[1]] 198 | poem = poem .. real_char 199 | --[[ 200 | if opt.verbose == 1 or real_char ~= '$' then 201 | io.write(real_char) 202 | end]]-- 203 | if real_char ~= "，" and real_char ~= "。" then 204 | table.insert(furtherexclusion, real_char) 205 | end 206 | if opt.verbose == 1 then 207 | -- calculate the top 10 characters 208 | local prediction_sorted, sorted_char = torch.sort(probs, 1, true) 209 | for j = 1, topprint do 210 | io.write(' ') 211 | io.write(ivocab[sorted_char[j]]) 212 | io.write('/') 213 | io.write(string.format("%.2f", probs[sorted_char[j]])) 214 | end 215 | io.write("\n") 216 | end 217 | if real_char == '$' then 218 | break 219 | end 220 | 221 | -- forward the rnn for next character 222 | local lst = protos.rnn:forward{prev_char, unpack(current_state)} 223 | current_state = {} 224 | for i=1,state_size do table.insert(current_state, lst[i]) end 225 | prediction = lst[#lst] -- last element holds the log probabilities 226 | end 227 | if string.sub(poem,#poem, #poem) == '$' then 228 | io.write(string.sub(poem,0, #poem-1) .. '。') 229 | poemCount = poemCount +1 230 | io.write('\n') io.flush() 231 | if poemCount == opt.length then 232 | break 233 | end 234 | end 235 | end 236 | 237 | -------------------------------------------------------------------------------- /util/CharSplitLMMinibatchLoader.lua: -------------------------------------------------------------------------------- 1 | 2 | -- Modified from https://github.com/oxford-cs-ml-2015/practical6 3 | -- the modification included support for train/val/test splits 4 | 5 | local CharSplitLMMinibatchLoader = {} 6 | CharSplitLMMinibatchLoader.__index = CharSplitLMMinibatchLoader 7 | 8 | function CharSplitLMMinibatchLoader.create(data_dir, batch_size, seq_length, split_fractions) 9 | -- split_fractions is e.g. {0.9, 0.05, 0.05} 10 | 11 | local self = {} 12 | setmetatable(self, CharSplitLMMinibatchLoader) 13 | 14 | local input_file = path.join(data_dir, 'input.txt') 15 | local vocab_file = path.join(data_dir, 'vocab.t7') 16 | local tensor_file = path.join(data_dir, 'data.t7') 17 | 18 | -- fetch file attributes to determine if we need to rerun preprocessing 19 | local run_prepro = false 20 | if not (path.exists(vocab_file) or path.exists(tensor_file)) then 21 | -- prepro files do not exist, generate them 22 | print('vocab.t7 and data.t7 do not exist. Running preprocessing...') 23 | run_prepro = true 24 | else 25 | -- check if the input file was modified since last time we 26 | -- ran the prepro. if so, we have to rerun the preprocessing 27 | local input_attr = lfs.attributes(input_file) 28 | local vocab_attr = lfs.attributes(vocab_file) 29 | local tensor_attr = lfs.attributes(tensor_file) 30 | if input_attr.modification > vocab_attr.modification or input_attr.modification > tensor_attr.modification then 31 | print('vocab.t7 or data.t7 detected as stale. Re-running preprocessing...') 32 | run_prepro = true 33 | end 34 | end 35 | if run_prepro then 36 | -- construct a tensor with all the data, and vocab file 37 | print('one-time setup: preprocessing input text file ' .. input_file .. '...') 38 | CharSplitLMMinibatchLoader.text_to_tensor(input_file, vocab_file, tensor_file) 39 | end 40 | 41 | print('loading data files...') 42 | local data = torch.load(tensor_file) 43 | self.vocab_mapping = torch.load(vocab_file) 44 | 45 | -- cut off the end so that it divides evenly 46 | local len = data:size(1) 47 | if len % (batch_size * seq_length) ~= 0 then 48 | print('cutting off end of data so that the batches/sequences divide evenly') 49 | data = data:sub(1, batch_size * seq_length 50 | * math.floor(len / (batch_size * seq_length))) 51 | end 52 | 53 | -- count vocab 54 | self.vocab_size = 0 55 | for _ in pairs(self.vocab_mapping) do 56 | self.vocab_size = self.vocab_size + 1 57 | end 58 | 59 | -- self.batches is a table of tensors 60 | print('reshaping tensor...') 61 | self.batch_size = batch_size 62 | self.seq_length = seq_length 63 | 64 | local ydata = data:clone() 65 | ydata:sub(1,-2):copy(data:sub(2,-1)) 66 | ydata[-1] = data[1] 67 | self.x_batches = data:view(batch_size, -1):split(seq_length, 2) -- #rows = #batches 68 | self.nbatches = #self.x_batches 69 | self.y_batches = ydata:view(batch_size, -1):split(seq_length, 2) -- #rows = #batches 70 | assert(#self.x_batches == #self.y_batches) 71 | 72 | -- lets try to be helpful here 73 | if self.nbatches < 50 then 74 | print('WARNING: less than 50 batches in the data in total? Looks like very small dataset. You probably want to use smaller batch_size and/or seq_length.') 75 | end 76 | 77 | -- perform safety checks on split_fractions 78 | assert(split_fractions[1] >= 0 and split_fractions[1] <= 1, 'bad split fraction ' .. split_fractions[1] .. ' for train, not between 0 and 1') 79 | assert(split_fractions[2] >= 0 and split_fractions[2] <= 1, 'bad split fraction ' .. split_fractions[2] .. ' for val, not between 0 and 1') 80 | assert(split_fractions[3] >= 0 and split_fractions[3] <= 1, 'bad split fraction ' .. split_fractions[3] .. ' for test, not between 0 and 1') 81 | if split_fractions[3] == 0 then 82 | -- catch a common special case where the user might not want a test set 83 | self.ntrain = math.floor(self.nbatches * split_fractions[1]) 84 | self.nval = self.nbatches - self.ntrain 85 | self.ntest = 0 86 | else 87 | -- divide data to train/val and allocate rest to test 88 | self.ntrain = math.floor(self.nbatches * split_fractions[1]) 89 | self.nval = math.floor(self.nbatches * split_fractions[2]) 90 | self.ntest = self.nbatches - self.nval - self.ntrain -- the rest goes to test (to ensure this adds up exactly) 91 | end 92 | 93 | self.split_sizes = {self.ntrain, self.nval, self.ntest} 94 | self.batch_ix = {0,0,0} 95 | 96 | print(string.format('data load done. Number of data batches in train: %d, val: %d, test: %d', self.ntrain, self.nval, self.ntest)) 97 | collectgarbage() 98 | return self 99 | end 100 | 101 | function CharSplitLMMinibatchLoader:reset_batch_pointer(split_index, batch_index) 102 | batch_index = batch_index or 0 103 | self.batch_ix[split_index] = batch_index 104 | end 105 | 106 | function CharSplitLMMinibatchLoader:next_batch(split_index) 107 | if self.split_sizes[split_index] == 0 then 108 | -- perform a check here to make sure the user isn't screwing something up 109 | local split_names = {'train', 'val', 'test'} 110 | print('ERROR. Code requested a batch for split ' .. split_names[split_index] .. ', but this split has no data.') 111 | os.exit() -- crash violently 112 | end 113 | -- split_index is integer: 1 = train, 2 = val, 3 = test 114 | self.batch_ix[split_index] = self.batch_ix[split_index] + 1 115 | if self.batch_ix[split_index] > self.split_sizes[split_index] then 116 | self.batch_ix[split_index] = 1 -- cycle around to beginning 117 | end 118 | -- pull out the correct next batch 119 | local ix = self.batch_ix[split_index] 120 | if split_index == 2 then ix = ix + self.ntrain end -- offset by train set size 121 | if split_index == 3 then ix = ix + self.ntrain + self.nval end -- offset by train + val 122 | return self.x_batches[ix], self.y_batches[ix] 123 | end 124 | 125 | -- *** STATIC method *** 126 | function CharSplitLMMinibatchLoader.text_to_tensor(in_textfile, out_vocabfile, out_tensorfile) 127 | local timer = torch.Timer() 128 | 129 | print('loading text file...') 130 | local f = torch.DiskFile(in_textfile) 131 | local rawdata = f:readString('*a') -- NOTE: this reads the whole file at once 132 | f:close() 133 | 134 | -- create vocabulary if it doesn't exist yet 135 | print('creating vocabulary mapping...') 136 | -- record all characters to a set 137 | local unordered = {} 138 | local len = 0 139 | -- code snippets taken from http://lua-users.org/wiki/LuaUnicode 140 | for char in string.gfind(rawdata, "([%z\1-\127\194-\244][\128-\191]*)") do 141 | if not unordered[char] then unordered[char] = true end 142 | len = len + 1 143 | end 144 | -- sort into a table (i.e. keys become 1..N) 145 | local ordered = {} 146 | for char in pairs(unordered) do ordered[#ordered + 1] = char end 147 | table.sort(ordered) 148 | -- invert `ordered` to create the char->int mapping 149 | local vocab_mapping = {} 150 | for i, char in ipairs(ordered) do 151 | vocab_mapping[char] = i 152 | end 153 | -- construct a tensor with all the data 154 | print('putting data into tensor...') 155 | local data = torch.ShortTensor(len) -- store it into 1D first, then rearrange 156 | local pos = 1 157 | for char in string.gfind(rawdata, "([%z\1-\127\194-\244][\128-\191]*)") do 158 | data[pos] = vocab_mapping[char] 159 | pos = pos + 1 160 | end 161 | 162 | -- save output preprocessed files 163 | print('saving ' .. out_vocabfile) 164 | torch.save(out_vocabfile, vocab_mapping) 165 | print('saving ' .. out_tensorfile) 166 | torch.save(out_tensorfile, data) 167 | end 168 | 169 | return CharSplitLMMinibatchLoader 170 | 171 | -------------------------------------------------------------------------------- /util/LineSplitLoader.lua: -------------------------------------------------------------------------------- 1 | 2 | -- Modified from https://github.com/oxford-cs-ml-2015/practical6 3 | -- the modification included support for train/val/test splits 4 | 5 | local LineSplitLoader = {} 6 | LineSplitLoader.__index = LineSplitLoader 7 | 8 | function LineSplitLoader.create(data_dir, batch_size, seq_length, split_fractions) 9 | -- split_fractions is e.g. {0.9, 0.05, 0.05} 10 | 11 | local self = {} 12 | setmetatable(self, LineSplitLoader) 13 | 14 | local input_file = path.join(data_dir, 'input.txt') 15 | local vocab_file = path.join(data_dir, 'vocab.t7') 16 | local tensor_file = path.join(data_dir, 'data.t7') 17 | 18 | -- fetch file attributes to determine if we need to rerun preprocessing 19 | local run_prepro = false 20 | if not (path.exists(vocab_file) or path.exists(tensor_file)) then 21 | -- prepro files do not exist, generate them 22 | print('vocab.t7 and data.t7 do not exist. Running preprocessing...') 23 | run_prepro = true 24 | else 25 | -- check if the input file was modified since last time we 26 | -- ran the prepro. if so, we have to rerun the preprocessing 27 | local input_attr = lfs.attributes(input_file) 28 | local vocab_attr = lfs.attributes(vocab_file) 29 | local tensor_attr = lfs.attributes(tensor_file) 30 | if input_attr.modification > vocab_attr.modification or input_attr.modification > tensor_attr.modification then 31 | print('vocab.t7 or data.t7 detected as stale. Re-running preprocessing...') 32 | run_prepro = true 33 | end 34 | end 35 | if run_prepro then 36 | -- construct a tensor with all the data, and vocab file 37 | print('one-time setup: preprocessing input text file ' .. input_file .. '...') 38 | LineSplitLoader.text_to_tensor(input_file, vocab_file, tensor_file) 39 | end 40 | 41 | print('loading data files...') 42 | local data = torch.load(tensor_file) 43 | self.vocab_mapping = torch.load(vocab_file) 44 | 45 | -- count vocab 46 | self.vocab_size = 0 47 | for _ in pairs(self.vocab_mapping) do 48 | self.vocab_size = self.vocab_size + 1 49 | end 50 | 51 | -- self.batches is a table of tensors 52 | print('reshaping tensor...') 53 | self.batch_size = batch_size 54 | local len = data:size(1) 55 | if ( len % batch_size ) ~= 0 then 56 | local newlength = math.floor(len / batch_size) * batch_size 57 | print("cutting off the end of data to make batch split evenly, total length " .. newlength) 58 | data = data:sub(1, newlength) 59 | end 60 | 61 | local ydata = data:sub(1,-1,2,-1):clone() 62 | -- ydata:sub(1,-2):copy(data:sub(2,-1)) 63 | -- ydata[-1] = data[1] 64 | data = data:sub(1,-1, 1,-2) 65 | assert(ydata:size(2) == data:size(2)) 66 | self.x_batches = data:split(batch_size, 1) -- #rows = #batches 67 | self.nbatches = #self.x_batches 68 | self.y_batches = ydata:split(batch_size, 1) -- #rows = #batches 69 | assert(#self.x_batches == #self.y_batches) 70 | 71 | -- lets try to be helpful here 72 | if self.nbatches < 50 then 73 | print('WARNING: less than 50 batches in the data in total? Looks like very small dataset. You probably want to use smaller batch_size and/or seq_length.') 74 | end 75 | 76 | -- perform safety checks on split_fractions 77 | assert(split_fractions[1] >= 0 and split_fractions[1] <= 1, 'bad split fraction ' .. split_fractions[1] .. ' for train, not between 0 and 1') 78 | assert(split_fractions[2] >= 0 and split_fractions[2] <= 1, 'bad split fraction ' .. split_fractions[2] .. ' for val, not between 0 and 1') 79 | assert(split_fractions[3] >= 0 and split_fractions[3] <= 1, 'bad split fraction ' .. split_fractions[3] .. ' for test, not between 0 and 1') 80 | if split_fractions[3] == 0 then 81 | -- catch a common special case where the user might not want a test set 82 | self.ntrain = math.floor(self.nbatches * split_fractions[1]) 83 | self.nval = self.nbatches - self.ntrain 84 | self.ntest = 0 85 | else 86 | -- divide data to train/val and allocate rest to test 87 | self.ntrain = math.floor(self.nbatches * split_fractions[1]) 88 | self.nval = math.floor(self.nbatches * split_fractions[2]) 89 | self.ntest = self.nbatches - self.nval - self.ntrain -- the rest goes to test (to ensure this adds up exactly) 90 | end 91 | 92 | self.split_sizes = {self.ntrain, self.nval, self.ntest} 93 | self.batch_ix = {0,0,0} 94 | 95 | print(string.format('data load done. Number of data batches in train: %d, val: %d, test: %d', self.ntrain, self.nval, self.ntest)) 96 | collectgarbage() 97 | return self 98 | end 99 | 100 | function LineSplitLoader:reset_batch_pointer(split_index, batch_index) 101 | batch_index = batch_index or 0 102 | self.batch_ix[split_index] = batch_index 103 | end 104 | 105 | function LineSplitLoader:next_batch(split_index) 106 | if self.split_sizes[split_index] == 0 then 107 | -- perform a check here to make sure the user isn't screwing something up 108 | local split_names = {'train', 'val', 'test'} 109 | print('ERROR. Code requested a batch for split ' .. split_names[split_index] .. ', but this split has no data.') 110 | os.exit() -- crash violently 111 | end 112 | -- split_index is integer: 1 = train, 2 = val, 3 = test 113 | self.batch_ix[split_index] = self.batch_ix[split_index] + 1 114 | if self.batch_ix[split_index] > self.split_sizes[split_index] then 115 | self.batch_ix[split_index] = 1 -- cycle around to beginning 116 | end 117 | -- pull out the correct next batch 118 | local ix = self.batch_ix[split_index] 119 | if split_index == 2 then ix = ix + self.ntrain end -- offset by train set size 120 | if split_index == 3 then ix = ix + self.ntrain + self.nval end -- offset by train + val 121 | return self.x_batches[ix], self.y_batches[ix] 122 | end 123 | 124 | -- *** STATIC method *** 125 | function LineSplitLoader.text_to_tensor(in_textfile, out_vocabfile, out_tensorfile) 126 | local timer = torch.Timer() 127 | 128 | print('loading text file...') 129 | local f = torch.DiskFile(in_textfile) 130 | local rawdata = f:readString('*a') -- NOTE: this reads the whole file at once 131 | f:close() 132 | 133 | -- create vocabulary if it doesn't exist yet 134 | print('creating vocabulary mapping...') 135 | -- record all characters to a set 136 | local unordered = {} 137 | local len = 0 138 | local linecount = 0 139 | local maxCharCount = 0 140 | local currentCharCount = 0 141 | -- code snippets taken from http://lua-users.org/wiki/LuaUnicode 142 | for char in string.gfind(rawdata, "([%z\1-\127\194-\244][\128-\191]*)") do 143 | if not unordered[char] then unordered[char] = true end 144 | if char == "\n" then 145 | linecount = linecount + 1 146 | if currentCharCount > maxCharCount then 147 | -- excluding the newline character 148 | maxCharCount = currentCharCount-1 149 | end 150 | currentCharCount = 0 151 | else 152 | currentCharCount = currentCharCount + 1 153 | end 154 | len = len + 1 155 | end 156 | -- sort into a table (i.e. keys become 1..N) 157 | local ordered = {} 158 | for char in pairs(unordered) do ordered[#ordered + 1] = char end 159 | table.sort(ordered) 160 | -- invert `ordered` to create the char->int mapping 161 | local vocab_mapping = {} 162 | for i, char in ipairs(ordered) do 163 | vocab_mapping[char] = i 164 | end 165 | -- construct a tensor with all the data 166 | print('putting data into tensor with dimension' .. linecount .. maxCharCount) 167 | -- NOTE: hack, my current data is one letter short on padding, so adding the s padding 168 | -- local data = torch.ShortTensor(len) -- store it into 1D first, then rearrange 169 | local data = torch.ShortTensor(linecount, maxCharCount+1) 170 | local pos = 1 171 | local linecount = 0 172 | local currentCharCount = 0 173 | -- code snippets taken from http://lua-users.org/wiki/LuaUnicode 174 | for char in string.gfind(rawdata, "([%z\1-\127\194-\244][\128-\191]*)") do 175 | if char == "\n" then 176 | -- more hack 177 | if ( currentCharCount ~= maxCharCount+1 ) then 178 | data[linecount+1][currentCharCount+1] = vocab_mapping['s'] 179 | end 180 | linecount = linecount + 1 181 | currentCharCount = 0 182 | else 183 | -- print(currentCharCount+1 .. ' ' .. pos .. ' ' .. len) 184 | data[linecount+1][currentCharCount+1] = vocab_mapping[char] 185 | currentCharCount = currentCharCount + 1 186 | end 187 | pos = pos + 1 188 | end 189 | 190 | -- save output preprocessed files 191 | print('saving ' .. out_vocabfile) 192 | torch.save(out_vocabfile, vocab_mapping) 193 | print('saving ' .. out_tensorfile) 194 | torch.save(out_tensorfile, data) 195 | end 196 | 197 | return LineSplitLoader 198 | 199 | -------------------------------------------------------------------------------- /util/OneHot.lua: -------------------------------------------------------------------------------- 1 | 2 | local OneHot, parent = torch.class('OneHot', 'nn.Module') 3 | 4 | function OneHot:__init(outputSize) 5 | parent.__init(self) 6 | self.outputSize = outputSize 7 | -- We'll construct one-hot encodings by using the index method to 8 | -- reshuffle the rows of an identity matrix. To avoid recreating 9 | -- it every iteration we'll cache it. 10 | self._eye = torch.eye(outputSize) 11 | end 12 | 13 | function OneHot:updateOutput(input) 14 | self.output:resize(input:size(1), self.outputSize):zero() 15 | if self._eye == nil then self._eye = torch.eye(self.outputSize) end 16 | self._eye = self._eye:float() 17 | local longInput = input:long() 18 | self.output:copy(self._eye:index(1, longInput)) 19 | return self.output 20 | end 21 | -------------------------------------------------------------------------------- /util/PaddedLineSplitLoader.lua: -------------------------------------------------------------------------------- 1 | 2 | -- Modified from https://github.com/oxford-cs-ml-2015/practical6 3 | -- the modification included support for train/val/test splits 4 | 5 | local PaddedLineSplitLoader = {} 6 | PaddedLineSplitLoader.__index = PaddedLineSplitLoader 7 | local vocab_size = 2000 8 | -- local vocab_size = 3 9 | -- * unknown 10 | -- ^ begin 11 | -- $ stop 12 | 13 | function PaddedLineSplitLoader.create(data_dir, batch_size, seq_length, split_fractions) 14 | -- split_fractions is e.g. {0.9, 0.05, 0.05} 15 | 16 | local self = {} 17 | setmetatable(self, PaddedLineSplitLoader) 18 | 19 | local input_file = path.join(data_dir, 'input.txt') 20 | local vocab_file = path.join(data_dir, 'vocab.t7') 21 | local tensor_file = path.join(data_dir, 'data.t7') 22 | 23 | -- fetch file attributes to determine if we need to rerun preprocessing 24 | local run_prepro = false 25 | if not (path.exists(vocab_file) or path.exists(tensor_file)) then 26 | -- prepro files do not exist, generate them 27 | print('vocab.t7 and data.t7 do not exist. Running preprocessing...') 28 | run_prepro = true 29 | else 30 | -- check if the input file was modified since last time we 31 | -- ran the prepro. if so, we have to rerun the preprocessing 32 | local input_attr = lfs.attributes(input_file) 33 | local vocab_attr = lfs.attributes(vocab_file) 34 | local tensor_attr = lfs.attributes(tensor_file) 35 | if input_attr.modification > vocab_attr.modification or input_attr.modification > tensor_attr.modification then 36 | print('vocab.t7 or data.t7 detected as stale. Re-running preprocessing...') 37 | run_prepro = true 38 | end 39 | end 40 | if run_prepro then 41 | -- construct a tensor with all the data, and vocab file 42 | print('one-time setup: preprocessing input text file ' .. input_file .. '...') 43 | PaddedLineSplitLoader.text_to_tensor(input_file, vocab_file, tensor_file) 44 | end 45 | 46 | print('loading data files...') 47 | local data = torch.load(tensor_file) 48 | self.vocab_mapping = torch.load(vocab_file) 49 | 50 | -- count vocab 51 | self.vocab_size = 0 52 | for _ in pairs(self.vocab_mapping) do 53 | self.vocab_size = self.vocab_size + 1 54 | end 55 | 56 | -- self.batches is a table of tensors 57 | print('reshaping tensor...') 58 | self.batch_size = batch_size 59 | local len = data:size(1) 60 | if ( len % batch_size ) ~= 0 then 61 | local newlength = math.floor(len / batch_size) * batch_size 62 | print("cutting off the end of data to make batch split evenly, total length " .. newlength) 63 | data = data:sub(1, newlength) 64 | end 65 | 66 | local ydata = data:sub(1,-1,2,-1):clone() 67 | -- ydata:sub(1,-2):copy(data:sub(2,-1)) 68 | -- ydata[-1] = data[1] 69 | data = data:sub(1,-1, 1,-2) 70 | assert(ydata:size(2) == data:size(2)) 71 | self.x_batches = data:split(batch_size, 1) -- #rows = #batches 72 | self.nbatches = #self.x_batches 73 | self.y_batches = ydata:split(batch_size, 1) -- #rows = #batches 74 | assert(#self.x_batches == #self.y_batches) 75 | 76 | -- lets try to be helpful here 77 | if self.nbatches < 50 then 78 | print('WARNING: less than 50 batches in the data in total? Looks like very small dataset. You probably want to use smaller batch_size and/or seq_length.') 79 | end 80 | 81 | -- perform safety checks on split_fractions 82 | assert(split_fractions[1] >= 0 and split_fractions[1] <= 1, 'bad split fraction ' .. split_fractions[1] .. ' for train, not between 0 and 1') 83 | assert(split_fractions[2] >= 0 and split_fractions[2] <= 1, 'bad split fraction ' .. split_fractions[2] .. ' for val, not between 0 and 1') 84 | assert(split_fractions[3] >= 0 and split_fractions[3] <= 1, 'bad split fraction ' .. split_fractions[3] .. ' for test, not between 0 and 1') 85 | if split_fractions[3] == 0 then 86 | -- catch a common special case where the user might not want a test set 87 | self.ntrain = math.floor(self.nbatches * split_fractions[1]) 88 | self.nval = self.nbatches - self.ntrain 89 | self.ntest = 0 90 | else 91 | -- divide data to train/val and allocate rest to test 92 | self.ntrain = math.floor(self.nbatches * split_fractions[1]) 93 | self.nval = math.floor(self.nbatches * split_fractions[2]) 94 | self.ntest = self.nbatches - self.nval - self.ntrain -- the rest goes to test (to ensure this adds up exactly) 95 | end 96 | 97 | self.split_sizes = {self.ntrain, self.nval, self.ntest} 98 | self.batch_ix = {0,0,0} 99 | 100 | print(string.format('data load done. Number of data batches in train: %d, val: %d, test: %d', self.ntrain, self.nval, self.ntest)) 101 | collectgarbage() 102 | return self 103 | end 104 | 105 | function PaddedLineSplitLoader:reset_batch_pointer(split_index, batch_index) 106 | batch_index = batch_index or 0 107 | self.batch_ix[split_index] = batch_index 108 | end 109 | 110 | function PaddedLineSplitLoader:next_batch(split_index) 111 | if self.split_sizes[split_index] == 0 then 112 | -- perform a check here to make sure the user isn't screwing something up 113 | local split_names = {'train', 'val', 'test'} 114 | print('ERROR. Code requested a batch for split ' .. split_names[split_index] .. ', but this split has no data.') 115 | os.exit() -- crash violently 116 | end 117 | -- split_index is integer: 1 = train, 2 = val, 3 = test 118 | self.batch_ix[split_index] = self.batch_ix[split_index] + 1 119 | if self.batch_ix[split_index] > self.split_sizes[split_index] then 120 | self.batch_ix[split_index] = 1 -- cycle around to beginning 121 | end 122 | -- pull out the correct next batch 123 | local ix = self.batch_ix[split_index] 124 | if split_index == 2 then ix = ix + self.ntrain end -- offset by train set size 125 | if split_index == 3 then ix = ix + self.ntrain + self.nval end -- offset by train + val 126 | return self.x_batches[ix], self.y_batches[ix] 127 | end 128 | 129 | -- *** STATIC method *** 130 | function PaddedLineSplitLoader.text_to_tensor(in_textfile, out_vocabfile, out_tensorfile) 131 | local timer = torch.Timer() 132 | 133 | print('loading text file...') 134 | local f = torch.DiskFile(in_textfile) 135 | local rawdata = f:readString('*a') -- NOTE: this reads the whole file at once 136 | f:close() 137 | 138 | -- create vocabulary if it doesn't exist yet 139 | print('creating vocabulary mapping...') 140 | -- record all characters to a set 141 | local unordered = {} 142 | local len = 0 143 | local linecount = 0 144 | local maxCharCount = 0 145 | local currentCharCount = 0 146 | -- code snippets taken from http://lua-users.org/wiki/LuaUnicode 147 | for char in string.gfind(rawdata, "([%z\1-\127\194-\244][\128-\191]*)") do 148 | if char == "\n" then 149 | linecount = linecount + 1 150 | if currentCharCount > maxCharCount then 151 | -- excluding the newline character 152 | maxCharCount = currentCharCount-1 153 | end 154 | currentCharCount = 0 155 | else 156 | if not unordered[char] then 157 | unordered[char] = 1 158 | else 159 | unordered[char] = unordered[char]+1 160 | end 161 | currentCharCount = currentCharCount + 1 162 | end 163 | len = len + 1 164 | end 165 | -- sort into a table (i.e. keys become 1..N) 166 | local ordered = {} 167 | --for char in pairs(unordered) do ordered[#ordered + 1] = char end 168 | for char, count in spairs(unordered, function(t,a,b) return t[b] < t[a] end) do 169 | ordered[#ordered + 1] = char 170 | end 171 | -- XXX what is this? 172 | -- table.sort(ordered) 173 | -- invert `ordered` to create the char->int mapping 174 | local vocab_mapping = {} 175 | local vocab_mapping_withu = {} 176 | local real_vocab_size = 1 177 | local vocab_occurance = 0 178 | for i, char in ipairs(ordered) do 179 | if i <= vocab_size then 180 | vocab_mapping_withu[char] = i 181 | vocab_mapping[char] = i 182 | vocab_occurance = vocab_occurance + unordered[char] 183 | real_vocab_size = real_vocab_size + 1 184 | else 185 | vocab_mapping_withu[char] = real_vocab_size 186 | end 187 | end 188 | if real_vocab_size > vocab_size then 189 | print("Shrinked vocab size to " .. vocab_size .. ' from ' .. #ordered .. ' by replacing low occurance with *') 190 | print("vocab coverage character percentage" .. (vocab_size/#ordered) .. ' occurrance count ' .. vocab_occurance .. ' / ' .. len .. '(' .. (vocab_occurance/len) .. ')') 191 | end 192 | vocab_mapping['*'] = real_vocab_size 193 | vocab_mapping['^'] = real_vocab_size+1 194 | vocab_mapping['$'] = real_vocab_size+2 195 | -- construct a tensor with all the data 196 | print('putting data into tensor with dimension' .. linecount ..' line ' .. maxCharCount .. 'characters') 197 | -- NOTE: hack, my current data is one letter short on padding, so adding the s padding 198 | -- local data = torch.ShortTensor(len) -- store it into 1D first, then rearrange 199 | local data = torch.ShortTensor(linecount, maxCharCount+2) 200 | local pos = 1 201 | local linecount = 0 202 | local currentCharCount = 1 203 | -- code snippets taken from http://lua-users.org/wiki/LuaUnicode 204 | for char in string.gfind(rawdata, "([%z\1-\127\194-\244][\128-\191]*)") do 205 | if char == "\n" then 206 | -- more hack 207 | for p = currentCharCount, maxCharCount+2 do 208 | data[linecount+1][p] = vocab_mapping['$'] 209 | end 210 | linecount = linecount + 1 211 | currentCharCount = 1 212 | else 213 | if currentCharCount == 1 then 214 | data[linecount+1][1] = vocab_mapping['^'] 215 | end 216 | -- print(currentCharCount+1 .. ' ' .. pos .. ' ' .. len) 217 | data[linecount+1][currentCharCount+1] = vocab_mapping_withu[char] 218 | currentCharCount = currentCharCount + 1 219 | end 220 | pos = pos + 1 221 | end 222 | 223 | -- save output preprocessed files 224 | print('saving ' .. out_vocabfile) 225 | torch.save(out_vocabfile, vocab_mapping) 226 | print('saving ' .. out_tensorfile) 227 | torch.save(out_tensorfile, data) 228 | end 229 | 230 | function spairs(t, order) 231 | -- collect the keys 232 | local keys = {} 233 | for k in pairs(t) do keys[#keys+1] = k end 234 | 235 | -- if order function given, sort by it by passing the table and keys a, b, 236 | -- otherwise just sort the keys 237 | if order then 238 | table.sort(keys, function(a,b) return order(t, a, b) end) 239 | else 240 | table.sort(keys) 241 | end 242 | 243 | -- return the iterator function 244 | local i = 0 245 | return function() 246 | i = i + 1 247 | if keys[i] then 248 | return keys[i], t[keys[i]] 249 | end 250 | end 251 | end 252 | 253 | return PaddedLineSplitLoader 254 | 255 | -------------------------------------------------------------------------------- /util/misc.lua: -------------------------------------------------------------------------------- 1 | 2 | -- misc utilities 3 | 4 | function clone_list(tensor_list, zero_too) 5 | -- utility function. todo: move away to some utils file? 6 | -- takes a list of tensors and returns a list of cloned tensors 7 | local out = {} 8 | for k,v in pairs(tensor_list) do 9 | out[k] = v:clone() 10 | if zero_too then out[k]:zero() end 11 | end 12 | return out 13 | end -------------------------------------------------------------------------------- /util/model_utils.lua: -------------------------------------------------------------------------------- 1 | 2 | -- adapted from https://github.com/wojciechz/learning_to_execute 3 | -- utilities for combining/flattening parameters in a model 4 | -- the code in this script is more general than it needs to be, which is 5 | -- why it is kind of a large 6 | -- https://groups.google.com/forum/#!topic/torch7/4f_wMQ6G2io 7 | 8 | require 'torch' 9 | local model_utils = {} 10 | function model_utils.combine_all_parameters(...) 11 | --[[ like module:getParameters, but operates on many modules ]]-- 12 | 13 | -- get parameters 14 | local networks = {...} 15 | local parameters = {} 16 | local gradParameters = {} 17 | for i = 1, #networks do 18 | local net_params, net_grads = networks[i]:parameters() 19 | 20 | if net_params then 21 | for _, p in pairs(net_params) do 22 | parameters[#parameters + 1] = p 23 | end 24 | for _, g in pairs(net_grads) do 25 | gradParameters[#gradParameters + 1] = g 26 | end 27 | end 28 | end 29 | 30 | local function storageInSet(set, storage) 31 | local storageAndOffset = set[torch.pointer(storage)] 32 | if storageAndOffset == nil then 33 | return nil 34 | end 35 | local _, offset = unpack(storageAndOffset) 36 | return offset 37 | end 38 | 39 | -- this function flattens arbitrary lists of parameters, 40 | -- even complex shared ones 41 | local function flatten(parameters) 42 | if not parameters or #parameters == 0 then 43 | return torch.Tensor() 44 | end 45 | local Tensor = parameters[1].new 46 | 47 | local storages = {} 48 | local nParameters = 0 49 | for k = 1,#parameters do 50 | local storage = parameters[k]:storage() 51 | if not storageInSet(storages, storage) then 52 | storages[torch.pointer(storage)] = {storage, nParameters} 53 | nParameters = nParameters + storage:size() 54 | end 55 | end 56 | 57 | local flatParameters = Tensor(nParameters):fill(1) 58 | local flatStorage = flatParameters:storage() 59 | 60 | for k = 1,#parameters do 61 | local storageOffset = storageInSet(storages, parameters[k]:storage()) 62 | parameters[k]:set(flatStorage, 63 | storageOffset + parameters[k]:storageOffset(), 64 | parameters[k]:size(), 65 | parameters[k]:stride()) 66 | parameters[k]:zero() 67 | end 68 | 69 | local maskParameters= flatParameters:float():clone() 70 | local cumSumOfHoles = flatParameters:float():cumsum(1) 71 | local nUsedParameters = nParameters - cumSumOfHoles[#cumSumOfHoles] 72 | local flatUsedParameters = Tensor(nUsedParameters) 73 | local flatUsedStorage = flatUsedParameters:storage() 74 | 75 | for k = 1,#parameters do 76 | local offset = cumSumOfHoles[parameters[k]:storageOffset()] 77 | parameters[k]:set(flatUsedStorage, 78 | parameters[k]:storageOffset() - offset, 79 | parameters[k]:size(), 80 | parameters[k]:stride()) 81 | end 82 | 83 | for _, storageAndOffset in pairs(storages) do 84 | local k, v = unpack(storageAndOffset) 85 | flatParameters[{{v+1,v+k:size()}}]:copy(Tensor():set(k)) 86 | end 87 | 88 | if cumSumOfHoles:sum() == 0 then 89 | flatUsedParameters:copy(flatParameters) 90 | else 91 | local counter = 0 92 | for k = 1,flatParameters:nElement() do 93 | if maskParameters[k] == 0 then 94 | counter = counter + 1 95 | flatUsedParameters[counter] = flatParameters[counter+cumSumOfHoles[k]] 96 | end 97 | end 98 | assert (counter == nUsedParameters) 99 | end 100 | return flatUsedParameters 101 | end 102 | 103 | -- flatten parameters and gradients 104 | local flatParameters = flatten(parameters) 105 | local flatGradParameters = flatten(gradParameters) 106 | 107 | -- return new flat vector that contains all discrete parameters 108 | return flatParameters, flatGradParameters 109 | end 110 | 111 | function model_utils.clone_many_times(net, T) 112 | local clones = {} 113 | 114 | local params, gradParams 115 | if net.parameters then 116 | params, gradParams = net:parameters() 117 | if params == nil then 118 | params = {} 119 | end 120 | end 121 | 122 | local paramsNoGrad 123 | if net.parametersNoGrad then 124 | paramsNoGrad = net:parametersNoGrad() 125 | end 126 | 127 | local mem = torch.MemoryFile("w"):binary() 128 | mem:writeObject(net) 129 | 130 | -- this is definitely not pointing to the same address. 131 | for t = 1, T do 132 | -- We need to use a new reader for each clone. 133 | -- We don't want to use the pointers to already read objects. 134 | local reader = torch.MemoryFile(mem:storage(), "r"):binary() 135 | local clone = reader:readObject() 136 | reader:close() 137 | 138 | if net.parameters then 139 | local cloneParams, cloneGradParams = clone:parameters() 140 | local cloneParamsNoGrad 141 | for i = 1, #params do 142 | cloneParams[i]:set(params[i]) 143 | cloneGradParams[i]:set(gradParams[i]) 144 | collectgarbage() 145 | end 146 | if paramsNoGrad then 147 | cloneParamsNoGrad = clone:parametersNoGrad() 148 | for i =1,#paramsNoGrad do 149 | cloneParamsNoGrad[i]:set(paramsNoGrad[i]) 150 | end 151 | end 152 | end 153 | 154 | clones[t] = clone 155 | collectgarbage() 156 | end 157 | 158 | mem:close() 159 | return clones 160 | end 161 | 162 | return model_utils 163 | -------------------------------------------------------------------------------- /yunjiao/YunjiaoLoader.lua: -------------------------------------------------------------------------------- 1 | require 'lfs' 2 | local YunjiaoLoader = {} 3 | YunjiaoLoader.__index = YunjiaoLoader 4 | -- data: http://baike.baidu.com/view/584090.htm?fromtitle=%E4%B8%83%E5%BE%8B&fromid=96270&type=syn#2_2 5 | -- 平水韵: https://zh.wikisource.org/zh-hans/%E5%B9%B3%E6%B0%B4%E9%9F%BB 6 | -- 0: whatever 7 | -- 1: 平 8 | -- 2: 仄 9 | -- 3: 平韵脚 （近体诗只押平声韵） 10 | -- 9: , . 11 | local orig_rules = { 12 | -- 七绝 13 | {'0102213', 14 | '0211223', 15 | '0201122', 16 | '0102213'}, 17 | 18 | {'0102112', 19 | '0211223', 20 | '0201122', 21 | '0102213'}, 22 | 23 | {'0211223', 24 | '0102213', 25 | '0102112', 26 | '0211223'}, 27 | 28 | {'0201122', 29 | '0102213', 30 | '0102112', 31 | '0211223'}, 32 | 33 | -- 七律 34 | { 35 | '0102211','0211223', 36 | '0201122','0102213', 37 | '0102112','0211223', 38 | '0201122','0102213', 39 | }, 40 | { 41 | '0102112','0211223', 42 | '0201122','0102213', 43 | '0102112','0211223', 44 | '0201122','0102213', 45 | }, 46 | { 47 | '0211221','0102213', 48 | '0102112','0211223', 49 | '0201122','0102213', 50 | '0102112','0211223', 51 | }, 52 | { 53 | '0201122','0102213', 54 | '0102112','0211223', 55 | '0201122','0102213', 56 | '0102112','0211223', 57 | }, 58 | 59 | -- 五律 60 | {'02211','11223', 61 | '01122','02213', 62 | '02112','11223', 63 | '01122','02213', 64 | },{ 65 | '02112','11223', 66 | '01122','02213', 67 | '02112','11223', 68 | '01122','02213', 69 | },{ 70 | '11221','02213', 71 | '02112','11223', 72 | '01122','02213', 73 | '02112','11223', 74 | },{ 75 | '01122','02213', 76 | '02112','11223', 77 | '01122','02213', 78 | '02112','11223', 79 | }, 80 | -- 五绝 81 | {'02112','11223','01122','02213', 82 | },{ 83 | '02211','11223','01122','02213', 84 | },{ 85 | '01122','02213','02112','11223', 86 | },{ 87 | '11221','02213','02112','11223', 88 | } 89 | } 90 | 91 | local rules = {} 92 | for _, rule in ipairs(orig_rules) do 93 | table.insert(rules, table.concat(rule, '9')) 94 | end 95 | 96 | -- charToYun: 翩: {1, 20} 97 | -- yunToChar: {1 = {20={'翩'}} 98 | function YunjiaoLoader.create() 99 | local self = {} 100 | setmetatable(self, YunjiaoLoader) 101 | 102 | -- th contains path package, but it breaks when only uses luajit 103 | -- local ze_file = path.join(path.dirname(paths.thisfile()), 'ze.txt'); 104 | -- local ping_file = path.join(path.dirname(paths.thisfile()), 'ping.txt'); 105 | -- local yunjiao_file = path.join(path.dirname(paths.thisfile()), 'yunjiao.t7') 106 | local ze_file = paths.dirname(paths.thisfile()) .. '/ze.txt' 107 | local ping_file = paths.dirname(paths.thisfile()).. '/ping.txt' 108 | local yunjiao_file = paths.dirname(paths.thisfile()).. '/yunjiao.t7' 109 | 110 | -- fetch file attributes to determine if we need to rerun preprocessing 111 | local run_prepro = false 112 | if not file_exists(yunjiao_file) then 113 | -- prepro files do not exist, generate them 114 | print('yunjiao.t7 do not exist. Running preprocessing...') 115 | run_prepro = true 116 | else 117 | -- check if the input file was modified since last time we 118 | -- ran the prepro. if so, we have to rerun the preprocessing 119 | local input_attr = lfs.attributes(ping_file) 120 | local vocab_attr = lfs.attributes(yunjiao_file) 121 | local ze_attr = lfs.attributes(ze_file) 122 | if input_attr.modification > vocab_attr.modification 123 | or ze_attr.modification > vocab_attr.modification then 124 | print('yunjiao.t7 detected as stale. Re-running preprocessing...') 125 | run_prepro = true 126 | end 127 | end 128 | if run_prepro then 129 | -- construct a tensor with all the data, and vocab file 130 | print('one-time setup: preprocessing input text file ...') 131 | YunjiaoLoader.text_to_yunjiao(ping_file, ze_file, yunjiao_file) 132 | end 133 | 134 | self.mapping = torch.load(yunjiao_file) 135 | 136 | collectgarbage() 137 | return self 138 | end 139 | 140 | function YunjiaoLoader:getYunjiao(char) 141 | if not self.mapping['charToYun'][char] then 142 | error("out of range char " .. char) 143 | end 144 | local idx = self.mapping['charToYun'][char] 145 | return self.mapping['yunToChar'][idx[1]][idx[2]] 146 | end 147 | 148 | function YunjiaoLoader:getPingze(char) 149 | if not self.mapping['charToYun'][char] then 150 | error("out of range char " .. char) 151 | end 152 | return self.mapping['charToYun'][char][1] 153 | end 154 | 155 | function YunjiaoLoader:getPingzeVocab(pingze) 156 | return flatten(self.mapping['yunToChar'][pingze]) 157 | end 158 | 159 | -- returns nil if there is no possible extension 160 | -- returns 9 if sentence needs to be finished 161 | function YunjiaoLoader:getNextPingzes(poem) 162 | if #poem == 0 then 163 | return {0} 164 | else 165 | local poemPingze = {} 166 | for char in string.gfind(poem, "([%z\1-\127\194-\244][\128-\191]*)") do 167 | table.insert(poemPingze, self:getPingze(char)) 168 | end 169 | local validPingze = {} 170 | for _, rule in ipairs(rules) do 171 | if YunjiaoLoader.matchRule(rule, poemPingze) then 172 | local n = #poemPingze+1 173 | validPingze[string.sub(rule,n,n)] = true 174 | end 175 | end 176 | local pingzeList = {} 177 | for key, _ in pairs(validPingze) do 178 | table.insert(pingzeList, key) 179 | end 180 | return pingzeList 181 | end 182 | end 183 | 184 | function YunjiaoLoader:belongsTo(pingzeList, char) 185 | local pingze = tonumber(self:getPingze(char)) 186 | for _, _p in pairs(pingzeList) do 187 | p = tonumber(_p) 188 | if p == 0 and pingze ~= 9 then 189 | return true 190 | end 191 | if p == 3 then 192 | p = 1 193 | end 194 | if pingze == p then 195 | return true 196 | end 197 | end 198 | return false 199 | end 200 | 201 | -- *** STATIC method *** 202 | function YunjiaoLoader.text_to_yunjiao(ping_infile, ze_infile, out_yunjiaofile) 203 | local data = { 204 | charToYun= {}, 205 | yunToChar= { {}, {} }, 206 | } 207 | 208 | local rawdata ={} 209 | local f = torch.DiskFile(ping_infile) 210 | table.insert(rawdata, f:readString('*a')) -- NOTE: this reads the whole file at once 211 | f:close() 212 | local f = torch.DiskFile(ze_infile) 213 | table.insert(rawdata, f:readString('*a')) -- NOTE: this reads the whole file at once 214 | f:close() 215 | for pingze, content in ipairs(rawdata) do 216 | local yunIdx = 1 217 | for char in string.gfind(content, "([%z\1-\127\194-\244][\128-\191]*)") do 218 | if char == "\n" then 219 | yunIdx = yunIdx + 1 220 | else 221 | if not data['yunToChar'][pingze][yunIdx] then data['yunToChar'][pingze][yunIdx] = {} end 222 | table.insert(data['yunToChar'][pingze][yunIdx], char) 223 | data['charToYun'][char] = {pingze,yunIdx} 224 | end 225 | end 226 | end 227 | 228 | data['charToYun']["，"] = {9,1} 229 | data['charToYun']["。"] = {9,2} 230 | 231 | torch.save(out_yunjiaofile, data) 232 | return data 233 | end 234 | 235 | function YunjiaoLoader.matchRule(rule, pingzeList) 236 | if #pingzeList > #rule then 237 | return false 238 | end 239 | for i, p in ipairs(pingzeList) do 240 | local r = string.sub(rule, i, i) 241 | if r == '3' and tostring(p) == '1' then 242 | 243 | elseif r ~= '0' and r ~= tostring(p) then 244 | return false 245 | end 246 | end 247 | return true 248 | end 249 | 250 | 251 | function flatten(arr) 252 | local result = { } 253 | 254 | local function flatten(arr) 255 | for _, v in ipairs(arr) do 256 | if type(v) == "table" then 257 | flatten(v) 258 | else 259 | table.insert(result, v) 260 | end 261 | end 262 | end 263 | 264 | flatten(arr) 265 | return result 266 | end 267 | 268 | function unicodeLength(ustring) 269 | local count = 0 270 | for char in string.gfind(content, "([%z\1-\127\194-\244][\128-\191]*)") do 271 | count = count + 1 272 | end 273 | return count 274 | end 275 | 276 | function startWith(String,Start) 277 | return string.sub(String,1,string.len(Start))==Start 278 | end 279 | 280 | function file_exists(name) 281 | if type(name)~="string" then return false end 282 | return os.rename(name,name) and true or false 283 | end 284 | 285 | return YunjiaoLoader 286 | 287 | 288 | 289 | 290 | -------------------------------------------------------------------------------- /yunjiao/ping.txt: -------------------------------------------------------------------------------- 1 | 东同铜桐筒童僮瞳筒中衷忠虫冲终戎崇嵩菘弓躬宫融雄熊穹穷冯风枫丰充隆空公功工攻蒙濛笼聋珑洪红鸿虹丛翁聪骢鬃通蓬篷烘潼蒙胧䓖匆砻峒罿螽狨沣癃幪梦潀讧嵕豵涷曈鲖翀忡嵩肜芃酆麷釭饛雺瞢璁谼恫嵷逢蝀侗絧艟犝氃爞瀜窿悾曚朦罞懵咙昽豅庞艐膧同詷戙穜种盅鼨茙駥芎沨蘴汎珫倥玒冢髳艨襱洚稯鬷猣螉蝬酮绒渱 2 | 冬农宗锺钟龙舂松衡容蓉庸封胸雍浓重从逢缝踪茸峰蜂锋烽蛩筇慵恭供琮悰淙侬松茏凶墉镛佣溶镕醲秾蛬邛共憧鄘颙喁邕壅痈饔纵龚枞賨脓淞忪彸憃冲瑢葑匈凶汹禺雍噰廱丰𫓩銎懵蚣蹖榕犎跫恟灉襛蝩桻咚彤褣瞛橦 3 | 江杠矼釭扛厖尨哤駹窗枞𫓩邦缸降泷双艭庞逢腔撞幢桩淙洚橦茳娏憃嵕谾𪻐漎豇蛖垹梆跫悾韸 4 | 支枝移为垂吹陂碑奇宜仪皮儿离施知驰池规危夷师姿迟龟眉悲之芝时诗棋旗辞词期基疑姬丝司葵医帷思滋持随痴维卮麋螭麾墀弥慈遗肌脂雌披嬉尸狸炊湄篱兹差疲茨卑亏蕤陲骑曦歧岐谁斯私窥欹熙欺疵赀笞羁彝髭颐资糜饥衰锥姨楣夔祇涯伊蓍追缁箕椎罴罳篪釐萎匙凘脾坻嶷治骊妫飔尸綦怡尼漪累匜牺饴而鸱推縻璃祁绥逵咿巇酏𫄨羲羸肢骐訾狮嗤毗咨堕萁其醨粢雎睢漓蠡噫骓馗菑辎褵邳锜胝緌鳍迤蛇陴淇蜊漦媸淄丽牦弥筛纚厮氏痍榱娭壝齍蓠轙脽蕲耏嫠貔比椑僖鸃贻祺葹嘻㧑鹂瓷鹚铍琦骴洏洟骙唲嵋怩欙駓熹孜台蚩罹裨虒魑荽纰椸倕丕琪僛耆惟猗剂絁羇伾荠黧偲潍提醾埘魌犛鲕蓰祗禧峗庳居糍鬐栀澌踦戏锤蚳畸鵻戣劘褫椅胹榰埤跜磁腄栘崥錍嗺郿暆痿酾桵离梩謻貤貾㛤佳簃锱陑虽蚑摫郫仔觺嘻寅鄑蓷鲥麒茈委鍉鸸秠蜞頯蘼軝剞桋襹摛棰崎嵫胔褷隋箄眵黐邿齝蜲𠯠爢蛦娸觯樆姼柅榯鼒翍觜錤缌厜鉹趍鸤蘪柌犪秜耛怌泜澬跠黟瓻恞峓鄈沩逶藣騩蘲踟踑諆圮瓵覗洢倭爔桸劙宦嵯祎诐玼桤觭徛椔蠯罢俾岯帔枇笓毗琵貔霉禔呢狔倪嫘樏藟梨犁蔾缡漓乖机赍伎埼只蜘砥隹郗弛篪锤槌蛳莳孖孳耔祠玭鹾罳撕濉宜宧诒迤眙崖嬴唯隗 5 | 微薇晖辉辉徽挥翚韦围帏闱违霏菲妃騑绯飞非扉肥腓威祈旗畿机几讥矶鞿玑饥稀希晞衣依沂巍归祎诽淝痱欷豨楎餥厞蝛葳肵鐖叽鵗溰犩馡婓颀埼圻睎騩 6 | 鱼渔初书舒居裾车渠蕖余予誉舆馀胥狙锄疏蔬梳虚嘘徐猪闾庐驴诸除储如墟垆菹琚旟玙与畬疽苴樗摅于箊茹蛆且沮袪祛蜍挐榈胪糈砠淤潴阹胠妤帤篨雎谞蘧腒鐻鶋椐纾袽躇橥趄璩鴽滁屠筡藘𦈌歔锄磲醵据瑹龉蠩唹驉摴蝑雓籧槠鵌呿魖藇铻疋咀蒘蒢湑衙涂狳洳薯 7 | 虞愚娱隅刍无芜巫于盂臞衢儒濡襦须须株诛蛛殊铢瑜榆谀愉腴区驱躯朱珠趋扶符凫雏敷夫肤纡输枢厨俱驹模谟蒲胡湖瑚乎壶狐弧孤辜姑觚菰徒途涂荼图屠奴呼吾梧吴租卢鲈鑪芦苏酥乌枯粗都铺禺嵎诬竽雩吁盱瞿劬朐胊絇軥鼩𦈡需䝙殳俞逾窬觎揄萸臾歈渝岖蒌镂娄苻莩孚桴郛俘柎趺𫓧迂姝蹰拘摹酺蒱醐糊糊鹕酤鸪沽呱蛄菟駼砮膴幠鼯笯驽逋舻垆徂孥泸栌𫗦晡玗嚅鸆蚨诹娵裯母軱罛瘏郚玈痡毋杅邘訏芙幮喁龋颅轳釪旴哻句醹邾洙褕羭愈蝓慺𦝼稃泭罦麸枹隃膜嫫瓠箛橭呼鋘恶刳溇躣瑜杸荂芋姁欨呕驺媭咮趎氀阇喻睮纑鰅枸欋鸬臑獳跦牏侏龉葫怃澞禂瓿㛀盓陓救柧鵌嵞籚痀稣圬于誧怤懦帑楰稌麌鴸蒩坞炉匍铺紨浮稃酴镀拏砮垆岣呱骷滹謼猢氍昫茱硃毹孺殂洿呜邬钨蜈庑污瘐逾 8 | 齐蛴脐黎藜梨蠡黧妻萋凄凄堤羝鞮低氐诋磾稊题提荑缔锑绨𫘨禔鹈媞缇折篦鎞鸡稽笄枅兮奚嵇蹊傒徯騱鼷鹥黳倪齯霓猊鲵𫐐酰西栖犀澌嘶撕梯鼙椑膍批跻齑赍挤懠迷麛泥臡谿圭窐邽睽奎刲携畦觿蠵烓骊鹂緀凄桋睼褆繄儿霓橀暌聧嶲霋笓錍粞啼狴砒玭醍𫛸蹄麑闺鲑乩栖㶉巂 9 | 佳街鞋牌柴钗差涯阶偕谐骸排乖怀淮豺侪埋霾斋娲蜗娃哇皆荄喈揩蛙湝楷痎櫰槐鲑緺䯄啀俳 10 | 灰恢魁隈回徊槐枚梅媒煤瑰雷垒隤催摧堆陪杯醅嵬推开哀埃台苔该才材财裁来莱栽哉灾猜胎台孩豗虺悝洄莓禖缞崔裴培坏骀垓陔騋徕毸皑镅傀焞蓷欸絯崃郲诙煨脢锤頧胚桅唉鲐炱荄才邰颏能椳茴衃儓薹侅峐偎漼隗捼咍杯徘抔掊玫抬颓颓罍儡赅咳盔追捶菑偲鳃挼呆铠 11 | 真因茵辛新薪晨辰臣人仁神亲申伸绅身宾滨邻鳞麟珍瞋尘陈春津秦频蘋颦嚬银垠筠巾囷民缗贫淳醇纯唇伦纶轮沦匀旬巡驯钧均臻瑧榛姻闉宸寅嫔龂旻彬鹑皴遵循振甄禋岷谆椿询恂峋漘莘堙屯骃呻粼磷辚璘濒嚚罠笢闽豳逡踆畇侁歅填訚狺泯旼忞洵溱诜駪桭湮傧驎磷夤荀郇錞迍竣紃蓁輑礥侲籈諲麎娠柛琎螓纫蠙鄞縜麇奫箘鶞珣抡蜦窀僎鷷袀姺甡墐畛潾嶙瞵眴斌兟氤 12 | 文闻纹云氛分纷芬焚坟群裙君军勤斤筋勋薰曛熏醺𫄸荤耘云芸棼汾𣸣枌雰员欣芹殷沄昕黂缊煴幩蕡焄纭郧縜豮妘羵鼖饙宭臐獯皲蒑殷勤慬廑瘽菫垠龂狺鄞訚雯衯蝹涢筼沄魵羒轒鼢蕲贲𪉃炘颁棻秎䜣辉听辒玟蚊汶氲 13 | 元原源鼋园援辕垣烦繁蕃樊翻幡暄萱喧冤言轩藩魂浑裈温孙门尊存蹲敦墩暾屯豚村盆奔论坤昏婚阍痕根恩吞騵沅嫄湲媛膰蹯燔爰薠蘩袢矾幡墦翻轓番璠反谖貆啍焞埙鶱鸳宛掀鞬昆琨鲲缊扪荪飧惇芚贲仑惛跟垠蹇薞鷷驐抡軘榬蕴睧鞎𦈉甗犍靬杬羱芫蚖榞邧阮袁洹蠜笲晅咺眢鹓怨蜿樠鼲沄溷崑辒璊虋亹嶟繜炖蜳饨臀庉湓棔喷拫纯樽沌炖囤锟騉髡辉涽馄骞圈喧埙狲瘟猿蕰 14 | 寒韩翰丹殚单安难餐滩坛檀弹残干肝竿干阑栏澜兰看刊丸桓纨端湍酸团抟攒官观冠鸾銮栾峦欢宽盘蟠漫干汗郸叹摊姗珊玕奸貆刓剜漙慱棺驩讙钻磐鞶瘢镘鬗谩瞒潘啴跚羱剬胖弁豻箪瘅拦完瓛岏莞髋般磻拌掸驙汍芄綄巑攒敦倌繁曼馒鳗痑禅籣谰貒峘洹狻眢涫湾羉樠盘慲蹒颟墁拼耑檀叹剸团鱄襕娈杆干顸鼾邗犴汉梡萑鳣酸鞍皖脘 15 | 删潸关弯阛还环镮鬟锾圜班斑颁般蛮颜奸菅攀顽豻山鳏间蕑艰闲闲娴鹇悭孱潺殷斒斓湲纶眅憪擐轘跧扳瞷鬘黫讪澴靬患獌玢豩寰娴痫黰僝疝湾 16 | 前千阡笺鞯天坚肩贤弦弦烟燕莲怜田填钿年颠巅牵妍研眠渊涓蠲边笾编玄县泉迁仙鲜钱煎然延筵毡旃鳣膻禅蝉缠廛躔连联涟篇偏便绵全宣镌穿川缘鸢铅捐旋娟船涎鞭铨筌专砖圆员干虔愆骞权拳椽传焉跹芊溅舷咽零阗骈軿鹃綖埏𫗴甄邅挻梴鋋嘕瀍翩扁平櫋牷朘脧儇翾瓀沿还悁诠痊佺悛荃篿遄卷颧鬈挛弮惓燀戋幵豜千纯祅蜎媊仟湔枅蚿畋佃磌蹎滇汧胼蠙鼘諓蜒潺孱婵儃楩瑄蠉懁駽撋蝝璇箯颛跧湲犍褰蔫嫣褼鵳歅瘨岍缏骿癣郔莚驙澶单仚𩧴絟竣𩨃鄢篯沺楄焆鋗籼鬋狿鹯扇揎堧璇猭键蜷蜷棉乾跣萹蹁癫钘湮趼狷悬褊睊煽啴栴链梿鲢漹攓搴揵缗拴旋漩鱄剸堧嬛棬婘 17 | 萧箫挑貂刁凋雕雕迢条髫跳蜩苕调枭浇聊辽寥撩僚寮邀峣么宵消尧霄绡销超朝潮嚣樵谯骄娇焦蕉椒燋饶桡荛烧遥傜姚摇谣轺瑶韶昭招飙标杓瓢苗描猫要腰鸮乔桥侨妖夭漂飘翘翛祧佻恌徼膋鹩漻侥哓哨虈枵熇獢穚膲娆飖鳐愮陶熛麃儦瀌葽喓弨趫橇劭潇鲦骁飂獠憀料橑簝膮痟硝蛸魈歊𨱓鷮鹪繇摽窑珧铫鹞猺褕钊髟膘薸篻蛲峤轿彯荍荞憿镣豂嘹垚怮逍揱怊燎毊憔嗂鉊贆剽幧翲僄 18 | 肴巢交郊茅嘲钞包胶爻苞梢蛟庖匏坳敲胞抛鲛崤铙骹炮髾筲哮呶捎𫍢麃茭淆虓蛸弰泡烋媌磝怓旓跑墝聱筊咬啁教咆犛鞘罺漅詨剿轈刨罞髇嗃佼抓䴔姣樔謷嘐掊瓟訬窌鄛脬飑鸼枹稍鄗洨庨莦俙摎捊顟嵺鞄涍轇 19 | 豪毫操绦髦刀萄猱褒桃槽漕旄袍挠蒿涛皋号陶螯翱鳌敖曹遭糕篙羔高嘈搔毛艘滔骚韬缫膏牢醪逃濠绹劳簩艚鱽洮慅叨绸慆醄𩙫璈牦芼舠螬裯忉饕骜獒熬臊槄梼祹匋掺涝弢蟧翿淘尻鼛謷啕挑槔嚣臑捞嗥蜪嶆薅櫜咎軞騊峱溞 20 | 歌多罗河戈阿和波科柯陀娥蛾鹅萝荷何过磨螺禾窠哥娑驼佗沱峨那苛诃珂轲痾莎蓑梭婆魔讹坡颇瑳紽酡鮀瘥莪俄哦傩呵皤么薖涡窝茄迦伽牁磋傞跎鹾詑番碆菏蹉搓驮驒醝緺献囮嶓蝌捼睋㳡踒箩锅倭啰埚嵯劘硪枷矬簻锣堶些桫他拖鼍逻哪挪髁吪陂鄱摩唆痤锉垛骡挼蒫艖趖魦臡鸁瘸 21 | 麻花霞家茶华沙车牙蛇瓜斜邪芽嘉瑕纱鸦遮叉葩奢楂琶衙赊涯巴加耶嗟遐笳差蟆蛙哗虾拏豭葭髽茄挝呀罝阇枷哑娲爬杷蜗䯄爷芭鲨窊豝緺珈騢桠骅赮娃哇洼麚洼畬了苴艖污鴐笯蕸铊舥夸裟瘕些跏涂桠杈楂痂姱岈鋘蚆溠秅荂哆碬爹椰齖煆蒘奓幏笆桦颬划犌迦揶锻吾蒫鎈犘𪉊浾铔砗祖钯疤佘沙砂渣查檛侘琊鍜袈丫夸胯抓呱靴姱跁衩 22 | 阳杨扬香乡光昌堂章张王房芳长塘妆常凉霜藏场央泱鸯秧嫱狼床方浆觞梁娘庄黄仓皇装肪殇襄相湘缃厢箱创忘芒望尝偿鲿樯枪坊囊郎唐狂强肠康冈苍匡荒遑行妨棠翔良航飏倡伥羌庆姜僵姜疆橿苌粮穰将墙桑刚祥详洋旸徉佯粱量羊伤汤鲂樟彰漳璋猖铓商防筐煌篁隍凰徨蝗惶璜榔廊浪筜裆沧纲亢吭钢丧肓潢簧忙茫傍汪臧琅螂当珰庠裳昴鷞鄣障疡锵镗硠桁杭颃邙赃湟滂桹溏砀骦筤禳攘跄鸧螀瀼瓤枋螗抢戕螳踉杗眶炀钖稂菖铛洸阊蜣玱蹡勷纕彭蘉蒋斨亡殃芗堈嫜鲳礓瓖蔷喤玚慞镶鬤汸邡钫孀嶈洭搪莨苀磄趪𫗮汒凔彷艡劻膷眻綡恇莣鉠榶瑭锒胱雱磅膀螃痒两魉强镪怆飨往谎倘惝硠阆榜嗓恍旁 23 | 庚更羹坑盲横觥彭棚亨枪英瑛烹平评枰京惊荆明盟鸣荣莹兵兄卿生甥笙牲檠擎鲸迎行衡耕萌氓甍纮宏闳茎罂罃莺樱泓橙争筝清情晴精睛菁晶旌盈楹瀛嬴赢营婴缨贞成盛城诚呈程酲声征正钲轻名令并倾萦饧琼鹒赓黉锽喤祊輣搒撑瞠枪韺霙伧峥苹枨猩鼪勍珩蘅桁铿硁牼栩嵘丁嘤鹦铮琤砰怦绷伻弸轰訇鍧瞪蜻䴖籯茔璎桢撄赪柽蛏侦郕珵裎鲭顷惸嬛骍榜洺栟狞抨絣趟掁嫇藑蝾醟瀯巆坪泙鶁禜浤吰娙 24 | 青经泾形刑邢硎铏型陉娙亭庭堥霆莛蜓渟楟停宁钉玎仃馨星腥醒惺俜娉灵棂醽龄铃苓伶泠零玲舲翎鸰瓴囹聆听厅汀冥溟蓂螟铭瓶屏軿萍荧萤荥扃坰駉葶鼮町𫐉酃桯瞑暝嫇䌹侀钘綎娗筳 25 | 蒸烝承丞惩澂陵凌绫菱冰膺鹰应蝇绳渑乘塍升升胜兴绘凭仍兢矜征凝称偁登簦灯僧鬙崩增曾憎矰橧层能棱朋鹏堋弘鞃肱薨腾滕藤縢恒緪鲮崚輘冯憴鱦陾艿症鄫驓噌磳瞢蕄掤螣揯篜凌殑夌淜譝騬礽扔庱砅鬅苰誊漰陞缯甑澄藤楞罾峘嶒 26 | 尤邮优忧流斿旒留榴骝刘由油游猷悠攸牛修脩羞秋楸周州洲舟酬仇柔俦畴筹稠丘抽瘳湫遒收鸠不搜驺愁休囚辀求裘球仔浮谋牟眸侔矛侯猴喉讴沤鸥瓯楼娄陬偷头投钩沟鞲幽虬彪疣訧耰麀绸嚘镠遛飂浏鹠瘤秋鹙蝣犹莸輶啾揪酋犨赒售浟蹂揉救蒐叟廋溲邹搊𥬠貅庥咻泅䌷裯帱鯈啁球逑絿𨱇觩俅赇蜉桴罦罘麰蛑鍪篌糇鍭欧𦝼慺搂寠抠腧軥裒阄髅蝼璆兜句妯惆菆篝抔呦缑呕媮缪诹繇蓲偻枹樛艛鲰掫齁琈烰篓蒌鄾鶔鴀紑鸺摎脙馗鞻蟉髟抌宄蚰卣庮鍒鞣涑鸺调鸼棷诌怮龟瀌瞀区緅骰侜頄艽芣鄇彄 27 | 侵寻浔林霖临针箴斟沈砧深淫心琴禽擒钦衾吟今襟金音阴岑簪骎琳琛椹谌忱壬任纴霪蟫愔黔嵚歆禁喑喑森参参涔嵾芩灊燖淋郴鵀妊檎紟椮綅祲綝湛沉 28 | 覃潭谭驔昙参骖南柟男谙庵含涵函岚蚕篸探贪眈耽湛龛堪戡弇谈惔甘三酣篮柑聃坩蓝锬担唅郯谽泔邯馣蜬儋鬖蚶憨韽毵镡淦痰甝婪嵁甔藫渰暗庵颔諵鸩褴倓澹鐕啽舑餤橝蟫媕醰 29 | 盐檐廉帘嫌严占髯谦奁纤签瞻蟾炎添兼缣霑尖潜阎幨黏淹箝甜恬拈砭铦暹詹襜渐歼黔铃猒蒹蔪痁燅忺阽鹣磏觇帘沾佥綅憸苫锨蚺湉挦占蠊薕襳𧮪鬑针柟崦阉熸瀸灊鳒噞枮 30 | 咸咸函缄岩谗衔岩帆衫杉监凡馋巉镵芟喃嵌掺瑊劖碞諴儳欃搀毚颿缄麙𧮪髟縿獑锨严 31 | -------------------------------------------------------------------------------- /yunjiao/ze.txt: -------------------------------------------------------------------------------- 1 | 董动孔总笼澒汞桶蠓空嵷滃琫懵蓊拢唪洞挏蒙幪玤菶懂硐塕鬷埲侗唝翪 2 | 肿种踵宠陇垄拥壅茸氄重冢奉覂勇涌踊甬俑蛹恐拱珙栱蛬巩竦悚耸汹湩拲溶恟駷鲖軵輁冗怂捧埇涌 3 | 讲港棒蚌项缿玤傋耩 4 | 纸只咫諟是轵枳砥抵氏靡彼毁毁委诡傀髓絫妓掎绮觜此泚豸褫徙屣蓰髀尔迩弭弥婢庳侈弛豕紫捶棰揣企旨指视美訾否兕几姊匕比妣轨水藟嚭唯止市恃征喜己纪跪技蚁迤酏俾鄙簋晷匦宄子梓矢洧鲔雉死履垒诔揆癸沚趾芷畤以已苡似耜汜姒巳祀史使驶耳珥駬里理里李俚鲤枲起芑杞屺跂士仕栜俟涘戺始峙痔齿矣拟薿耻祉滓笫胏垝嶲舣锜蒍薳玼廌玺逦酾纚鞞敉芊哆姼庀跬頍秕机氿欙圮痞痔儗坻褆嶬花阤旎址址悝娌嗺壝佹匜剞踦耔佌崺讄秭秠倚被底痏岿蕊 5 | 尾鬼苇扆蚁卉虺几亹伟韪篚朏炜猪顗靴斐诽菲悱棐虮榧岂偯暐匪玮蜚蜰蘬唏 6 | 语圉圄御龉敔吕侣旅膂纻苎宁杼伫羜与予渚煮汝茹暑鼠黍杵处贮褚楮醑糈谞湑女籹许拒距炬虡巨秬苣所楚础阻俎沮举莒筥叙序绪𫚈藇屿墅衙峿稆梠癙著稰巨駏岠鐻濋咀跙苴榉讵柜溆纾去儢 7 | 麌雨羽禹宇舞父府鼓虎古股羖贾蛊土吐谱圃庾户树麈煦貐琥怙嵝蒟仵咻醹楰珇篓卤謱努弣罟肚妩沪龋枸斞冔邬鄅瞴蔖辅组乳弩补鲁橹橹竖腐卤数簿姥普拊每五庑斧聚午伍缕部柱矩武脯苦取抚浦主杜祖堵愈祜扈雇虏甫黼莆甒腑俯怃簠膴估诂盬牯瞽酤怒俣瑀祤喣踽窭楛稌浒诩栩寙炷拄剖鹉岵溥砮赌愈䉤伛偻蒌莽淦睹 8 | 荠礼体米启醴陛洗邸底诋抵抵柢坻弟悌娣递涕济蠡澧欐鳢泚綮棨髀祢徯媞癠眯弥醍缇鲚泲挤氐抵砥泥昵睨奶溪 9 | 蟹解骇买洒楷獬廌澥𫘤奶锴駴摆罢拐矮伙 10 | 贿悔改采彩彩海在罪宰醢载喂铠恺待怠殆倍猥隗磈嵬嶵磥蕾癗儡礧櫑錞腇采绐诒蓓鼐颏骀欸琲垲廆浼頠汇瘣漼璀每亥乃 11 | 轸敏允引尹尽忍准隼笋盾楯闵悯泯菌箘蚓靷纼诊眕畛胗紾哂肾脤膑牝冁赈窘蜃陨殒蠢蠢紧狁簨缜袗踳纯偆霣愍眹吮朕稹囷黾嶙 12 | 吻粉蕴愤隐谨近恽忿槿菫坋弅坟听龀刎殷鼢 13 | 阮远本晚苑返反阪损饭偃堰衮遁稳蹇幰巘楗揵婉菀蜿踠晼宛畹琬阃梱壸鲧悃捆辊绲鳟撙很恳垦畚圈盾刌绻鄢混沌鼹鰋蝘噂娩烜咺焜棍 14 | 旱暖管琯满短馆缓盥款懒伞卵散伴诞浣瓒断笴侃算缵暵蜑但酂衎脘坦袒亶秆窾粄悍懑纂篹痯悹趱 15 | 潸眼简版盏产限睅撰栈绾赧戁浐𡶴盏羼丳僝睆柬拣莞僩蝂眅钣輚馔皖板阪汕铲 16 | 铣善遣浅典转衍犬选冕辇免展茧辩辨篆勉翦卷显饯践眄喘藓软巘蹇演岘栈舛荈扁脔谳阐兖娈跣腆鲜戬铉吮辫件笕琏蝡撚泫墠墡单畎褊惼艑瑑蜓殄腼甗蚬贙俛缅沔湎趼键狝襺黾蒇辗搴蜎琄𪾢愐洗齴鬋戭燹筅癣狷燀鄟諓钱趁僤韅毨隽揃歂缱涊嵃幝撰剸耎鞬谝匾撰宴姺碥俴缏萹餮沴捵晛篯啴𫗴膳鳝䏝僎稨楩娩謰沇馻卷蜒剪谫颤鱄邅縯 17 | 筱小表鸟了晓少扰绕绕娆绍杪秒沼眇矫蓼皦皎了朓脁窱杳窅窈嬲袅袅袅皛窕挑掉湫肇旐駣鮡慓摽缥醥篻渺缈訬藐淼袑蟜挢娇譑𫏋褾标殍溔鷕悄愀缭僚麃昭夭佻燎赵兆 18 | 巧饱卯昴狡爪鲍挠搅绞拗茆佼姣炒泖媌铰筊瑵 19 | 皓宝藻早枣老好道稻造脑恼岛倒捣抱考燥埽嫂槁潦保葆堡褓鸨草皞昊浩颢镐鄗懆滈缲璪皂袄缫駣蚤澡薧灏栲媪蝹夭杲缟橑轑恅芺蓩栳套璅娼涝燠扫 20 | 哿火笴舸瑳亸哆柁沱我硪娜傩荷可坷轲左果裹蜾朵锁琐堕垛惰妥坐么裸蠃蓏跛簸颇叵祸伙輠颗砢鬌瘅堁那卵嫷娑脞埵爹惈婐椭陊嶞峨閜揣 21 | 马下者野雅瓦寡社写泻夏冶也鲊把贾假舍赭斝厦嘏槚惹若踝姐哆哑灺且瘕銙扯疋姹閜髁庌厊輠痄洒玛笆舍喏槎娅鲑剐打耍那 22 | 养痒鞅怏泱像象橡仰朗奖桨敞昶氅枉迋颡强穰沆崵荡簜惘磢昉放仿驵蚃两緉帑谠傥曩杖响掌党想榜爽广享丈仗幌晃莽漭襁襁纺蒋攘盎鲞坱𣗋潒脏苍皝长上网荡壤瀁赏仿罔辋蟒滉吭榥灢蚢饷磉魍抢恍慌蛘厂慷犷向璗瓬曭楖蒡奘 23 | 梗影景井岭领境警请屏饼永骋逞颍颖顷整静省幸颈郢猛炳瘿杏丙邴打哽绠秉鲠耿璟憬荇犷并皿冏煚靓矿艋蜢黾怲蛃窉鲠箵冷靖悜睛裎 24 | 迥炯茗挺梃艇铤町颋醒溟酊嫇脡褧娗冼庱珽刭莛等鼎顶泂诇婞侹胫肯颎泞拯涬酩 25 | 有酒首手口母后柳友妇斗狗久负厚叟走守绶右否丑受牖偶耦阜九后咎薮吼帚垢亩舅纽藕朽臼肘韭剖诱牡缶酉扣欧笱瓿黝蔀蹂取钮狃掊耇莠丑苟糗某玖拇纣纠嗾卣罶杻槱枸塿忸浏郈赳蚪䉤懰茆培滫醙擞嵝扣茩掫妵莥萯黈绺眑庮溇篓趣陡枓羑楺鲰琇蟉寿殴 26 | 寝饮锦品枕审甚廪衽饪稔禀葚沈凛懔噤沈谂淰腍踸瞫朕荏恁锓婶 27 | 感览掔榄胆澹憺啖坎惨憯敢颔暗窞黮萏歜撼毯菼紞椠贉晻椮菡黕喊揜黪澉顉眈寁嗿醰髧昝轗顄衴橄錾嵌欿赣 28 | 琰焰敛险俭检脸染掩点簟贬冉苒陕陕谄奄渐玷忝剡潋飐芡闪嗛歉慊溓隒猃黡魇扂 29 | 豏槛范减舰犯湛斩黯范掺阚喊淰轞笵滥黤歉瀺巉 30 | 送梦凤洞众瓮弄贡冻痛栋仲中讽恸鞚空控哢湩哄恫赣赗幪砻哄羾瞢詷絧衷涷淞蕻謥 31 | 宋重用颂诵统纵讼种综俸共供从缝葑壅雍封雺蠢疭 32 | 绛降巷蠢撞虹洚哄憧幢艟淙 33 | 寘置事地意志治思泪吏赐字义利器位戏至次累寺瑞智记异致备肆翠骑使试类弃饵媚鼻易辔坠醉议翅避笥帜粹侍谊师厕寄睡忌贰萃穗二帔臂嗣吹遂恣四骥季刺驷柶泗识痣志寐魅邃燧隧穟璲襚檖䍁睟牸植炽织饲食积忮被芰懿悸觊冀暨懻惎洎穊蔇愧匮鐀馈篑蒉恚比庇畀痹诐毖閟泌秘鸷贽挚觯胾踬渍迟埴祟豉珥衈咡刵示伺嗜自眦骴詈痢莉致轾譬彗蔧肄眙惴儗怼缢赑喂劓啻饎企晒勚眊膇为贲糒腻施鄪遗跂槌柲僿哆誋潩诒值柴樲髲出萎澌垝髊硾腄蚑翨掎樻缒蜼蚝貤累廙肂坒其异谇屣锤佽岿施庳孳睢騺懫司诿臮陂塈甀侐咥几近始术里欬跸瑟德邲疐倕庛胔离嫘袘觖倚踦委伪嬖跛懥视谥率槜鐩彗悴瘁质稚雉莅痵饐曀喟愧馈柜秘费莳畤椔驶亟薏莿徛濞薙屃骳 34 | 未味气贵费沸尉畏慰蔚魏纬胃渭汇谓讳卉毅溉既禨旡蔇衣饩熂黖忾忥欷塈概诽芾痱屝痱蜚翡罻黂气暨 35 | 御处去虑誉署据驭曙助絮著豫翥箸恕与遽疏庶诅预倨茹语踞锯狙沮劾洳滪饫淤蓣胠醵除鐻瘀棜鑢呿怚悆藇礜如鸒悇椐女讵欤楚嘘 36 | 遇路潞辂赂璐露鹭树度渡赋布步固痼锢素具数怒务雾鹜骛附兔故顾雇句墓暮慕募注注澍驻炷胙阼裕误悟寤晤住戍库护頀濩屦诉蠹妒惧趣娶铸胯傅付谕妪芋捕哺污忤厝措错醋鲋祔仆赙赴酺恶互孺怖煦寓冱酤瓠输吐铺謼溯屡嗉塑跗斁捂簬呴瞿驱讣菟鉒馵姁婺枑吁属作嫭酗雨获秅镀涸傃圃戽饇驸足抪苦𫗦蚹蒟咮获 37 | 霁制计势世丽岁卫济第艺惠慧币桂滞际厉涕契弊毙帝蔽敝髻锐戾裔袂系祭隶闭逝缀翳制替砌细税婿例誓筮蕙偈诣砺励瘗噬继脆谛系睿毳剂曳蒂睇憩彗睨堄醊贳穧沴枻逮柢禘芮掣傺豷蓟穄妻挤眦禊弟墆遾釱鷩蹛寱契题砅蛎潎禲睥筀嚏盭竁枘彗递遰愒猘鳜粝疠嬖蹶齐棣说彘曀离荔汭泥蜕赘俪揭帨唳薙泄殪娣澨哜刿薜懘懠呓濞捩羿谜轪鵱杕憓蜧欐蘻痸綟箅畷缔鳀甈悷嘒浙嫕晢楴淠忕切踶䗖蟪槥 38 | 泰会带外盖大濑赖籁蔡害最贝霭蔼沛艾兑柰奈绘桧脍浍狯侩襘旝郐禬癐荟磕壒太汰汏釱轪癞粝霈蜕濊翙哕酹狈茷祋藾愒昧旆眛沬梖駾赉丐 39 | 卦挂懈廨隘卖画瘥派债怪坏诫戒界介芥械薤拜快迈话败稗晒噫瘵届疥玠瀣湃瞆惫铩杀夬哙嘬虿喝解祭齘蒯蒉犗餲繲絓粺价喟狯砦诖劢繣篑呗欬寨 40 | 队内塞爱辈佩代退载碎态背秽菜对废诲晦昧碍戴贷配妹喙溃黛赉吠逮岱埭肺溉耒慨忾块缋乂碓赛刈耐悖暧倅晬淬敦愦阓铠硙颣焙在再欬孛鄁瑁痗茷薉柿憝礧酹瀣薆叆镦睐徕襶裁叇儗采回顪焠栽悖北拔縡薱劾谇脢采悔废鼐眛朏珮 41 | 震信印进润阵镇填刃顺慎鬓晋骏闰峻衅振舜吝烬讯允仞轫殡傧迅瞬榇儭谆荩慭殣馑蔺浚徇殉赈觐畯馂摈葭琎酳仅牣认遴賮衬鬊瑾趁龀蕣韧讱侲汎蹸躏驎浚墐缙搢娠靷引瞵诊蜃瑱疢亲揗 42 | 问闻运晕韵训粪奋忿酝郡分紊汶偾愠焮靳近斤抆絻郓餫员缊璺拼隐蕰坋 43 | 愿论怨恨万饭献健寸困顿遁建劝宪蔓券钝闷逊嫩贩愿溷远巽噀曼喷艮敦坌慁绻郾褪畹楦堰圈 44 | 翰岸汉难断乱叹干观散畔旦算玩烂贯半案按炭汗赞赞漫冠灌爨窜幔粲灿璨换焕唤悍捍弹惮段看判叛腕涣奂绊惋雚钻缦锻旰闬瀚焊骭豻胖暵谰𫘣蒜罐瓘酂喭衎泮逭祼漶墁彖毈鴠干盰矸谩澜碬撺褖摊侃悹馆滩晏盥爟 45 | 谏雁患涧间宦晏慢办盼豢鷃栈惯赝輚串苋绽幻讪丱绾骭缦嫚谩汕疝瓣薍擐篡铲裥虥栅粯扮襻 46 | 霰殿面变箭战扇煽膳传见砚选院练链宴燕宴弮贱电荐绢彦掾甸便眷线倦羡堰奠恋啭眩钏蒨倩卞汴弁拼忭咽片禅谴绚谚缘颤擅授嫒瑷佃钿淀淀缮鄯狷罥睊煎旋瑱唁穿竁茜甗溅柬拣缠牵先剸炫袨炫眴善缱遣研嬿猭瞑汧填珔洊栫蚬𪾢贙趼狿娈鄄莚伣撰鬋眄諓衍榗辗转綪縓涀僆饯炼遍 47 | 啸笑照庙窍妙诏召劭邵要曜耀耀调钓吊叫嘂燎峤少徼眺诮料肖尿剽掉鹞粜藋噭轿窔朓脁僬烧疗釂漂醮铫骠茑爝趭慓绕摽娆獥摇窱葽鹩顤哨约僄艞嘹嬥裱俵趒熽莜跳嫽镣廖鞘峭俏悄帩剿饶獠彯票皭婊 48 | 效教貌校孝桡闹淖豹儤爆罩踔趠拗窖酵嗃袎稍乐效较钞疱敲恔笊礉棹觉珓窌胶 49 | 号帽报导盗操噪噪奥隩告诰暴好到蹈劳傲秏眊耄躁涝漕造冒悼纛焘倒骜瑁媢翿缟懊澳慥嫪奡趮抱虣膏犒郜芼凿氉埽祷墺瀑旄燠靠糙灶 50 | 个贺佐作逻坷轲驮大饿那些过和挫课堁唾播簸锉莝磨座坐破卧货磋涴左锉惰瘅譒奈 51 | 祃驾夜下谢榭罢夏暇霸灞嫁赦借藉炙蔗假化舍价射骂稼架诈亚娅罅跨麝怕讶诧嗄檽迓蜡胯帊柘崋姹贳弝泻砑靶乍桦杷埧坝卸唶鹧侘偌吓哑华话窊汊呀笮厍杈衩 52 | 漾上望相将状帐浪唱让旷壮放向仗畅量葬匠障谤尚涨饷样藏舫访贶养酱嶂抗当酿亢况脏瘴王纩鬯谅亮妄怆刱丧怅两圹宕伉忘傍砀恙吭炀飏张阆胀行广悢汤炕韔长创诳桁緉羕踼闶曏颃醠彷掠妨搒旺迋荡潢防怏偿荡盎仰瀁饷挡傥 53 | 敬命正令政性镜盛行圣咏姓庆映病柄郑劲竞净竟孟迸聘诤泳请倩禜硬凊靓檠晟獍怲更横醟榜迎娉敻轻并儆评邴证诇侦并侦盟 54 | 径定听胜磬应乘媵赠称罄邓甑胫莹证孕兴经甯醒廷锭庭顁饤钉靘暝滢烝剩剩凝嶝镫橙磴墱凳蹬堋 55 | 宥候堠就授售寿秀宿奏绣富兽漏陋守狩昼寇茂懋旧胄胄宙袖褎岫柚覆复救厩臭幼佑祐右侑囿豆脰窦逗溜溜瘤廇留构遘媾觏冓购透瘦漱镂贸鹫走副狖诟糅酎究凑谬缪籀疚炙畜雊鷇柩繇骤甃首皱绉戊句袤鼬僦瞀咮窌蹂姆沤姤廖腠蔟又鲎馏鹨辏逅蔻伏簉蜼槱收狃嗾鍑犹饾后油雺仆鞣后厚扣琇楱酘擩塯鍭吼僽𫐓绶读懤椆辐飂輶鄮楙𫍲偻 56 | 沁饮禁任荫谶浸祲谮鸩枕衽赁临渗喑揕维闯僸鵀妊噤紟吟罧深甚侺沈窨 57 | 勘暗滥啖担憾缆瞰琀憺绀阚三暂甔磡赣参澹淡憨瞰錾淦爁惭 58 | 艳剑念验赡店占敛厌滟爓潋垫欠椠窆僭酽坫幨砭餍噞猃殓苫煔掞痁盐沾兼念酓胁姭俺潜爁嬮忝 59 | 陷鉴监汎梵帆忏儳蘸韽阚谗镵剑欠淹站錎赚譀摲泛氾 60 | 屋木竹目服福禄榖熟谷肉族鹿腹菊陆轴逐牧伏宿读犊牍渎椟黩讟毂复粥肃育六缩哭幅斛戮仆畜蓄叔淑菽独卜馥沐速祝镞蹙筑穆睦啄覆鹜麹秃縠扑衄鬻燠澳辐瀑漉蔌恧洑鵩竺筑簇蔟暴掬箙濮鞫鞠掬郁矗复簏蓿塾朴蹴煜谡碌琭盝踘醭韣毓舳柚蝠昱菔辘朒慉踧樕稑夙蹜蝮彧𫗧柷匐淯觫鱐霂僇㱩俶摵缪輹螰蓼熇鵴澓槲觳剭莤蓫囿梀薁葍毣楘纀苜倏噈槭茯涑睩碡髑虙瘯逼颓副就摝 61 | 沃俗玉足曲粟烛属录箓辱狱绿毒局欲束鹄蜀促触续督赎笃浴酷缛瞩躅褥旭蓐欲顼梏纛蠋歜裻溽瘃跼挶輂勖醁渌逯騄喾牿襮鄏鹆告鋈熇仆 62 | 觉角桷捔翯玨较榷榷岳乐鸑浞灂穛斮娖朔数箾欶斲卓诼涿噣倬琢峃椓剥趵爆驳驳邈瞀儿眊雹謈懪瓝豰璞朴墣飑壳确悫埆觳硞浊擢鵫镯棹鸐濯幄喔偓药握渥搦踔逴荦学鸴齱 63 | 质日笔出室实疾术一乙壹吉秩密率律逸佚失漆栗毕恤恤蜜橘溢瑟膝匹述栗黜跸弼七叱卒虱悉谧术轶诘帙戍佶栉昵窒必侄蛭泌镒秫苾蟀嫉唧篥遹鹬筚骘佾怵繂珌锧帅崒潏礩聿姞抶驲郅桎庢铚挃晊耴泆沕繘踤茁紩璱獝飶尼堲柣蒺罼佖秷熚駜櫍铋觱鷅麜蛣衵咥汨汩蔤驈滭拮 64 | 物佛拂屈郁乞掘讫吃绂黻韨綍弗茀祓诎崛勿熨欻厥沕仡𨰿迄汔怫艴刜不屹肸芴岉黦菀咈倔沷尉蔚 65 | 月骨发阙越谒没伐罚卒竭窟笏钺歇发突忽勃蹶鹘筏厥蕨掘阀讷殁粤悖兀兀碣猝樾羯汨汩窣咄惚捽凸渤龁蝎滑刖𫐄劂崒腯孛纥浡暍矻淈鷢核麧饽瞂馞搰蟨柮棁抈撅鳜阏卼杌硉扤矹屼楬榾悖淴嗢泏堀朏胐椊扢抇狘猲愲艴曰堨讦钀 66 | 易达末阔活脱夺褐割沫拔葛闼渴拨豁括抹秣遏挞萨掇喝跋魃獭撮怛阏剌栝筈钹泼輵軷茇頞越斡剟嶭捋鞨鸹鹖毼暍鱍躠适𢫬攃袯齾猲濊佸葀犮羍笪鵽泧眓莌堨呾咄餲汱粝 67 | 黠札拔猾鹘八察杀刹轧辖刖蚻菝劼螖耻豽朒鸹戛秸嘎扴磍北揠蔱汃樧茁砎齾楬瞎獭刮錣鵽帕妠擖刷铩颉滑 68 | 屑节雪绝列烈结穴说血舌洁别缺热决铁灭折切拙裂悦辙诀泄咽噎杰哲彻鳖设啮劣碣掣谲玦截窃缬蠥缀阅埒讦餮瞥撇茢蛚臬闑媟昳臲锲耋抉挈洌捩楔蹩亵瓞襭绖蔑嵲陧捏篞醊茁竭契𫔎谳岊疖巀涅颉撷撤跌蔑浙鷩潎趹瞲篾蕝揲澈蛭揭垤孑孽凸闭阕锊齛薛绁楶瀎泬渫偈啜楬轶霓桀苶辍爇晰迭歠侄咥核惙呐洌嶭颲掇吷畷剟准棁拮蛣批橇絜觖 69 | 药薄恶略作乐落阁鹤爵弱约脚雀幕洛壑索郭博错跃若缚酌托削铎灼凿郤络鹊诺度萼橐漠钥著虐掠获泊搏龠崿锷藿嚼杓勺簙酪谑廓绰霍烁镬莫箨铄缴谔鄂亳恪箔攫涸汋彴疟爚镢鹗龠礿郝髆𪨗骆膜粕镆饦霩妁漷泺跞拓蠖镈格昨柝酢臛醵萚𫏋斫摸貉珞愕怍鞹柞垩笮玃膊鑮臄斮凿噱瘼爝箬蒻魄烙药堮焯攉鄗謞嗃熇厝噩咢泽袅碏矍硌各皵瞙矐躇芍婼躩踖踱沰靃剫戄岝鮥鄀燋迮逴泽鸙欂貜皭魠昔瀹婥渃却逽托跅雒爆簿寞腭鳄濩扩陌趵获 70 | 陌石客白泽伯宅策碧籍格役帛戟璧驿麦额柏魄积夕液册尺隙逆画百辟赤易革脊获翮屐适帻碛隔益栅窄核核舄掷责惜僻癖辟掖腋释舶拍索择磔摘射绎怿斥奕弈帟迫疫译昔瘠赫炙谪虢腊箦硕赜奭螫藉翟襞嗌穸砉祏亦鬲擗踖貘诉觡骼只鲫珀齸借膈啧扼踯埸蝪帼掴蹐婳峄斁绤席貊擘檗跖擿馘汐塉湱橶哑柞摭醳唶霡咋吓郤躄剌莫潟蓦襫蝈襗鼫耤厝霸霹佰坼拆檡拓喀假謋擭嚄迮蚱舴却剧脉薜梀摵涑䇲槭滆嗝厄厄扼轭划帼啯蕮碛膌鹡跖嗌睾燡蜴澼搦格虩圛迹 71 | 锡壁历历枥击绩勣笛敌滴镝檄激寂翟觌逖籴析溺觅摘狄荻鹢戚戚戚涤的菂吃甓霹沥雳苈愓裼踢剔緆砾栎轹皪鬲汨汩砉适嫡靮阋焱鵙蹢觋郦踧菥淅蜥籊吊霓鹝澼趯獥倜毄惄塓臭殈薂馰樀艗 72 | 职国德食蚀色力翼墨极息直得北黑侧饰贼刻则塞式轼域殖植敕饬棘惑默织匿亿臆忆特勒劾慝昃仄稷识逼克克蜮唧即拭弋陟测冒翊抑恻扐泐肋亟殛忒湜緎棫淢罭畟崱螣荝鷘阈嶷僰纆襋洫踣熄寔啬穑埴菔匐釴芅隿黓瀷屴犆恧轖鲫繶檍阞腷湢楅赩蜮樴幅杙愊副仂或蠈愎醷翌侐栻堲稄衋意涩稙值廙蕀堛艒鄎勀 73 | 缉辑戢立集邑急入泣湿习给十拾什龙及级粒揖汁笈蛰笠执隰汲吸唈絷葺褶潗苙伋岌翕歙濈裛浥熠槢潝霫悒廿挹馵岦钑蕺霵 74 | 合塔答纳榻阁杂腊蜡匝阖蛤衲沓榼鸽踏飒拓拉遝搭韐漯盍欱鞳唈靸钑馺趿阘誻軜溘嗑答姶涾鞜嚃卅嗒磕郃盒喋雥褡錔妠匼盖搕瞌闸塌遢蹋业邺胁胠怯抾袷跲笈腌噆邋劫衱浥裛 75 | 叶帖贴牒接猎妾蝶叠箧涉鬣捷颊楫摄蹑谍堞协侠荚晔厌惬勰睫浃笈慑慑蹀挟铗屧喋箑褶镊靥楪韘烨詟折裛讘鎑跕歙霅魇褋躐艓擸踕緁萐謵捻躞苶惵鍱滠衱婕聂梜椄菨獦倢鯜霎蛱 76 | 洽狭峡硖法甲业邺匣压鸭乏怯劫胁愶插锸歃押狎袷祫帢翣掏嶪喋夹恰眨呷胛萐箑柙郏鵊霅霎扱喋札擖跲嗋欱喢圔渫钾韐 77 | --------------------------------------------------------------------------------