├── .gitignore ├── Kconfig ├── Makefile ├── README ├── fault-injection.c ├── fault-injection.h └── tests ├── .gitignore ├── Makefile ├── test_fault_inject.c └── test_fault_inject.sh /.gitignore: -------------------------------------------------------------------------------- 1 | # 2 | # NOTE! Please use 'git ls-files -i --exclude-standard' 3 | # command after changing this file, to see if there are 4 | # any tracked files which get ignored after the change. 5 | # 6 | # Normal rules 7 | # 8 | .* 9 | *.o 10 | *.o.* 11 | *.out 12 | *.a 13 | *.s 14 | *.ko 15 | *.so 16 | *.so.dbg 17 | *.mod.c 18 | *.i 19 | *.lst 20 | *.symtypes 21 | *.order 22 | modules.builtin 23 | *.elf 24 | *.bin 25 | *.gz 26 | *.bz2 27 | *.lzma 28 | *.xz 29 | *.lzo 30 | *.patch 31 | *.gcno 32 | 33 | # 34 | # Top-level generic files 35 | # 36 | /tags 37 | /TAGS 38 | /linux 39 | /vmlinux 40 | /vmlinuz 41 | /System.map 42 | /stamp-building 43 | /build-arch-stamp 44 | /build-indep-stamp 45 | /configure-stamp 46 | 47 | # 48 | # git files that we don't want to ignore even it they are dot-files 49 | # 50 | !.gitignore 51 | !.mailmap 52 | 53 | # 54 | # Generated include files 55 | # 56 | include/config 57 | include/linux/version.h 58 | include/generated 59 | arch/*/include/generated 60 | 61 | # 62 | # Generated module tables 63 | Module.markers 64 | Module.symvers 65 | modules.order 66 | -------------------------------------------------------------------------------- /Kconfig: -------------------------------------------------------------------------------- 1 | config FAULT_INJECTION2 2 | tristate "Highly configurable fault-injection framework" 3 | depends on DEBUG_KERNEL && JUMP_LABEL && X86 4 | ---help--- 5 | Provide highly configurable fault-injection framework 6 | based on jump labels. For more details, see README 7 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | # 2 | # Makefile for fault-injection 3 | # 4 | 5 | obj-$(CONFIG_FAULT_INJECTION2) := fault-injection.o 6 | -------------------------------------------------------------------------------- /README: -------------------------------------------------------------------------------- 1 | Fault Injection 2 | =============== 3 | 4 | Author: Roman Penyaev 5 | Author: Roman Penyaev 6 | 7 | Introduction 8 | ------------ 9 | 10 | Fault injection is a framework which aims to help developers and testers 11 | to reproduce subtle bugs and to test thorougly error paths in the code. 12 | 13 | Firstly developer has to put special fault points in desired places all 14 | over the code, then those fault points can be configured in different 15 | ways and various faults can be injected. 16 | 17 | Currently fault injection supports three kind of faults: delays, errors 18 | and panics. 19 | 20 | Implementation Details 21 | ---------------------- 22 | 23 | Fault injection is based on jump labels, which means that big amount of 24 | fault points spread all over the code should not impact performance, 25 | because CPU will execute NOP if fault points are disabled. 26 | 27 | Fault points are statically compiled in and thus do not require memory 28 | allocation. Also, debugfs API is provided for flexible configuration 29 | from userspace. 30 | 31 | Generally speaking, fault point is a code branch which is executed only 32 | if fault point has been enabled. This code branch should be put by the 33 | developer and only she decides, according to the code logic, what should 34 | be executed if fault was injected. Let's take a real code example: 35 | 36 | mem = kmalloc(len, GFP_KERNEL); 37 | if (unlikely(!mem)) { 38 | pr_err("%s: allocation failed\n", __func__); 39 | return -ENOMEM; 40 | } 41 | 42 | Here it will be interesting to introduce an error instead of calling 43 | kmalloc and to test an error path. Using fault injection framework it 44 | can be done like this: 45 | 46 | + #define kmalloc(...) \ 47 | + (INJECT_FAULT() ? NULL : kmalloc(__VA_ARGS__)) 48 | + 49 | mem = kmalloc(len, GFP_KERNEL); 50 | if (unlikely(!mem)) { 51 | pr_err("%s: allocation failed\n", __func__); 52 | return -ENOMEM; 53 | } 54 | 55 | If this fault point was configured to inject an error kmalloc will not 56 | be executed and an error will be returned from the macro. 57 | 58 | In the example above compiler will optimize INJECT_FAULT, and by default 59 | NOP will be executed followed. When this fault point is configured to 60 | introduce an error, NOP instruction will be replaced with JMP instruction 61 | and slow path of the code will be executed. 62 | 63 | Internally fault point is represented as a static stateless entry compiled 64 | in __jump_table, so at any point of time we can list all the points, 65 | configure, enable or disable them. 66 | 67 | As was told, fault point is stateless, thus it does not know what kind 68 | offault it should inject, and that should be decided while doing 69 | configuration. That approach gives a lot of freedom to manipulate fault 70 | points and to configure them according to testing scenarios. 71 | 72 | 73 | Injecting faults in the code or macros API 74 | ------------------------------------------ 75 | 76 | Firstly fault injection must be registered by the module, in which 77 | it is supposed to be used: 78 | 79 | static struct fault_inject inj; 80 | 81 | fault_inject_register(&inj, THIS_MODULE); 82 | 83 | Fault injection framework provides three kinds of macros for fault 84 | injection: 85 | 86 | o INJECT_FAULT - 87 | Simplest one, returns an error if fault has been injected. 88 | 89 | Code example: 90 | + err = INJECT_FAULT(&inj, "MEM"); 91 | + if (unlikely(err)) 92 | + return err; 93 | 94 | o INJECT_FAULT_INT - 95 | Accepts function parameter, which will be executed if fault was 96 | not injected. In case of fault injection function will not be 97 | executed and error as integer will be returned. 98 | 99 | Code example: 100 | - err = do_useful_stuff(a, b, c); 101 | + err = INJECT_FAULT_INT(&inj, "DEV", do_useful_stuff(a, b, c)); 102 | if (unlikely(err)) 103 | return err; 104 | 105 | o INJECT_FAULT_PTR - 106 | This macro is almost the same as INJECT_FAULT_INT, but instead 107 | of integer representation of an error pointer will be returned. 108 | If error has happened it will be incapsulated inside a pointer. 109 | 110 | Code example: 111 | - ptr = do_useful_stuff(a, b, c); 112 | + ptr = INJECT_FAULT_PTR(&inj, "DEV", do_useful_stuff(a, b, c)); 113 | if (unlikely(IS_ERR(ptr))) 114 | return PTR_ERR(ptr); 115 | 116 | Second parameter for each INJECT_FAULT macro is a shost fault class 117 | name, which can be used for easy faults parsing. In the example 118 | above two fault classes were created "MEM" and "DEV". 119 | 120 | As was told in introduction those fault points can introduce three kind 121 | of faults: delays, errors or panics. What faults should be raised is 122 | decided on the configuration stage. 123 | 124 | 125 | Configuration from userspace or debugfs API 126 | ------------------------------------------- 127 | 128 | Each module has it's own fault injection configuration which is based on 129 | debugfs and is located here: /sys/kernel/debug/fault_inject/{MODULE_NAME}/. 130 | 131 | List of debugfs entries and their meaning: 132 | 133 | o list_fault_points [read only file] 134 | 135 | Reading from this file outputs the whole list of compiled in fault 136 | points, e.g. the output can be: 137 | 138 | fault group class address function+off/size file:line 139 | ----- ----- ---- 0xffffffffa068b0b4 init+0x0b4/0x5c6 main.c:2140 140 | ----- ----- ---- 0xffffffffa068b13a init+0x13a/0x5c6 main.c:2150 141 | DE--- 1 ---- 0xffffffffa061234b exit+0x12a/0x33 main.c:342 142 | -E--- 2 ---- 0xffffffffa042313c foo+0x124a/0x14a main.c:11 143 | 144 | 'fault' column - types of faults which are configured for this fault 145 | point, where D - delay, E - error, P - panic. 146 | 'group' column - the group to which the fault point belongs, where 147 | '-----' means fault point does not belong to any 148 | group. 149 | 'class' column - class of the fault. 150 | 'address' column - address in the code where fault point is placed. 151 | 'function' column - function name, offset and the size where fault 152 | point is placed. 153 | 'file' column - file and line of the current source file where 154 | fault point is placed. Keep in mind, that because 155 | of function inlining multiple different functions 156 | can have fault points with equal file name and line. 157 | 158 | Reading from this file will give you explicit information about all 159 | fault points for the module and their configuration, like to which 160 | group it belongs or what kinds of faults are enabled right now. 161 | 162 | o create_group [write only file] 163 | 164 | Writing some number to that file creates fault group with name corresponding 165 | to the number you have written. Fault group is an abstraction which unites 166 | variety of fault points which should be configured equally. So, basically 167 | configration is performed for fault group, not for each fault point, but 168 | fault group includes variety of fault points. E.g. you can use this command: 169 | 170 | # echo 0 > ./create_group 171 | 172 | If group already exists error will be returned. In case of success 173 | directory with group name will be created, i.e. 0/ in current example. 174 | 175 | The maximum possible amount of groups are limited to 256. 176 | 177 | o next_group [read only file] 178 | 179 | Reading from this file returns next free group number, e.g. 180 | 181 | # cat ./next_group 182 | 1 183 | 184 | That can be helpful to use from testing scripts in such command: 185 | 186 | # cat ./next_group > ./create_group 187 | 188 | Of course user should think about concurrent access. 189 | 190 | o delete_group [write only file] 191 | 192 | Writing group number to that file removes specified group. If group does 193 | not exist error will be returned. 194 | 195 | o {group_number}/ [directory] 196 | 197 | Fault group directory which has configuration files for this group. 198 | 199 | o {group_number}/list_fault_points [read only file] 200 | 201 | Reading from this file outputs the list of compiled in fault points 202 | included to the current group. By default the list is empty and contains 203 | the header only: 204 | 205 | # cat ./list_fault_points 206 | fault group class address function+off/size file:line 207 | 208 | o {group_number}/add_fault_points [write only file] 209 | 210 | Writing line with code address to that file will add fault point to this 211 | group. Basic requirements are: 212 | 213 | o line should have '\n' at the end 214 | o the total length of the line should not exceed 127 chars, i.e. 126 + \n 215 | o line should have any address in hex starting from 0x 216 | 217 | For example: 218 | 219 | # echo 0xffffffffa068b0b4 > ./add_fault_points 220 | 221 | or more convenient way to add all fault points (please note that we 222 | catting from 'list_fault_points' located in parent directory): 223 | 224 | # cat ../list_fault_points > ./add_fault_points 225 | 226 | or even filtering: 227 | 228 | # cat ../list_fault_points | grep main.c > ./add_fault_points 229 | 230 | If fault point already belongs to some group an error will be returned. 231 | If address is parsed but does not exist an error will be returned. 232 | 233 | o {group_number}/del_fault_points [write only file] 234 | 235 | Writing line with code address to that file will remove fault point 236 | from that group. Basic requirements and restrictions are similar to 237 | what is listed in in the description to add_fault_points file. 238 | 239 | For example: 240 | 241 | # cat ./list_fault_points | grep main.c | sponge ./del_fault_points 242 | 243 | BEWARE: use 'sponge' to soak up all the input before writing to 244 | del_fault_points, because in other case you will get an error. 245 | Why? Keep in mind that you are reading the fault points from 246 | the list and at the same time removing them from the same list. 247 | 'sponge' will help you! 248 | 249 | o {group_number}/delay/ [directory] 250 | 251 | Directory with delay configuration for this group of fault points. 252 | 253 | o delay_us [read write file, default value 0] 254 | 255 | How many us of delay should be introduced if fault is injected. 256 | Accepts integer value. 257 | 258 | o {group_number}/error/ [directory] 259 | 260 | Directory with error configuration for this group of fault points. 261 | 262 | o errors [read write file, default value is empty string] 263 | 264 | Writing comma separate list of errors to that file will introduce 265 | specified error on fault injection. Errors will be processed sequentially, 266 | using round-robin pattern. 267 | 268 | Reading from the file returns configured list of errors prefixed 269 | with minuses. 270 | 271 | For example writing: 272 | 273 | # echo -ENOSPC,-EAGAIN > ./errors 274 | 275 | or without minuses: 276 | 277 | # echo ENOSPC,EAGAIN > ./errors 278 | 279 | and reading them back: 280 | 281 | # cat ./errors 282 | -EAGAIN,-ENOSPC 283 | 284 | In case of parsing failure error will be returned. 285 | 286 | o {group_number}/panic/ [directory] 287 | 288 | Directory with panic configuration for this group of fault points. 289 | If enabled and execution steps on fault point the kernel will panic. 290 | 291 | o Common files for delay/, error/ panic configurations: 292 | 293 | o enable [read write file, default value 0] 294 | 295 | Writing 1 or 0 enables or disables respectively the configuration. 296 | Reading from that file tells you if that configuration is enabled 297 | or not. 298 | 299 | o hits [read only file, default value 0] 300 | 301 | Statistics value which tells how many times that group of fault 302 | points was hit. Keep in mind, that hitting does not mean fault 303 | injection, because we have other parameters like probability, 304 | interval or times. 305 | 306 | o injected [read only file, default value 0] 307 | 308 | Statistics value which tells how many times that group of fault 309 | points was really injected. So basically for group, which is 310 | configured to delay, that means how many times executed fault points 311 | was delayed, or for group, which was configured to introduce an error, 312 | that means exactly how many times error was returned. 313 | 314 | o times [read write file, default value -1] 315 | 316 | Specifies how many times failures may happen at most. 317 | A value of -1 means "no limit". 318 | Zero value is not accepted. 319 | 320 | o probability [read write file, default value 100] 321 | 322 | Likelihood of failure injection, in percent. 323 | Accepted values are in the range [1, 100]. 324 | 325 | For example, if probability is set to 50 the ratio of hits and 326 | injected should be close to 2. 327 | 328 | o interval [read write file, default value 1] 329 | 330 | Specifies the interval between failures. 331 | Zero value is not accepted. 332 | 333 | o task_filter [read write file, default value 0] 334 | 335 | The default value is 0, which means filtering by task is disabled. 336 | Any positive value limits failures to only processes indicated by 337 | /proc//make-it-fail==1. 338 | 339 | NOTE: This option is available only if CONFIG_FAULT_INJECTION is 340 | enabled for the kernel configuration. 341 | -------------------------------------------------------------------------------- /fault-injection.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Fault injection framework. 3 | * 4 | * Copyright (c) 2018 ProfitBricks GmbH. 5 | * Authors: Roman Pen 6 | * 7 | * This program is free software; you can redistribute it and/or 8 | * modify it under the terms of the GNU General Public License as 9 | * published by the Free Software Foundation, version 2 of the 10 | * License. 11 | */ 12 | 13 | #include 14 | #include 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | 22 | #define FAULT_INJECT_MODULE 23 | #include "fault-injection.h" 24 | 25 | #define FAULT_HEADER "fault" 26 | #define GROUP_HEADER "group" 27 | #define CLASS_HEADER "class" 28 | #define ADDR_HEADER "address" 29 | #define FUNC_HEADER "function+off/size" 30 | #define FILE_HEADER "file:line" 31 | #define FAULT_LINE_SZ 128 /* 126 letters, 1 newline, 1 null byte. 32 | * before changing anything check 33 | * 'format_fault_point' */ 34 | 35 | enum fault_type { 36 | DELAY_FAULT = 0, /* place first faults which do not generate 37 | * errors to keep probing all of them */ 38 | ERROR_FAULT, 39 | PANIC_FAULT, 40 | 41 | FAULTS_NUM /* should be the last */ 42 | }; 43 | 44 | const char fault_type_chars[FAULTS_NUM] = { 45 | 'D', 'E', 'P', 46 | }; 47 | 48 | const char *fault_type_strings[FAULTS_NUM] = { 49 | "delay", "error", "panic", 50 | }; 51 | 52 | struct fault_cfg { 53 | struct dentry *dentry; 54 | const char *name; 55 | enum fault_type type; 56 | bool enabled; 57 | bool task_filter; 58 | atomic64_t hits; 59 | atomic64_t injected; 60 | atomic_t times; 61 | unsigned short probability; 62 | unsigned short interval; 63 | struct fault_group *group; 64 | union { 65 | struct { 66 | unsigned int beg; 67 | unsigned int end; 68 | } delay_us; 69 | DECLARE_BITMAP(error_mask, FAULT_ERRORS_SZ); 70 | }; 71 | }; 72 | 73 | struct fault_group { 74 | struct fault_fs_ref fs_ref; 75 | struct fault_inject *inj; 76 | struct dentry *dentry; 77 | unsigned short id; 78 | atomic_t ref; 79 | struct list_head list; 80 | struct mutex lock; 81 | wait_queue_head_t wait; 82 | struct fault_cfg faults[FAULTS_NUM]; 83 | }; 84 | 85 | /* Keep in sync with asm constructor, see arch_static_branch */ 86 | struct fault_point { 87 | struct static_key key; /* always the first */ 88 | unsigned int magic; 89 | unsigned int line; 90 | const char *file; 91 | const char *class; 92 | struct fault_group *group; 93 | }; 94 | 95 | static inline void fault_fs_ref_init(struct fault_fs_ref *fs_ref) 96 | { 97 | atomic_set(&fs_ref->ref, 0); 98 | init_waitqueue_head(&fs_ref->wait); 99 | } 100 | 101 | static inline void *priv_from_dentry(struct dentry *dentry) 102 | { 103 | return dentry->d_inode->i_private; 104 | } 105 | 106 | static inline struct fault_fs_ref *ref_from_fault_inject( 107 | void *priv) 108 | { 109 | struct fault_inject *inj = priv; 110 | 111 | return &inj->fs_ref; 112 | } 113 | 114 | static inline struct fault_fs_ref *ref_from_fault_group( 115 | void *priv) 116 | { 117 | struct fault_group *group = priv; 118 | 119 | return &group->fs_ref; 120 | } 121 | 122 | static inline struct fault_fs_ref *ref_from_fault_cfg( 123 | void *priv) 124 | { 125 | struct fault_cfg *cfg = priv; 126 | 127 | return &cfg->group->fs_ref; 128 | } 129 | 130 | /** 131 | * fault_fs_ref_get() - Returns private pointer of dentry if dentry is still 132 | * hashed. 133 | * 134 | * Why do we need this? Because debugfs has stupid race when someone calls 135 | * debugfs_remove(), but another task performs file open and eventually 136 | * succeeds, but caller of debugfs_remove() is absolutely sure that dentry 137 | * was removed. Here we check under the lock that dentry is still hashed, 138 | * if so the reference is increased, so even after the lock is released 139 | * out data is protected by the reference counter. 140 | */ 141 | static void *fault_fs_ref_get( 142 | struct dentry *dentry, 143 | struct fault_fs_ref *(*conv)(void *)) 144 | { 145 | struct fault_fs_ref *fs_ref; 146 | void *priv; 147 | 148 | spin_lock(&dentry->d_lock); 149 | if (d_unhashed(dentry)) 150 | priv = NULL; 151 | else { 152 | priv = priv_from_dentry(dentry); 153 | fs_ref = conv(priv); 154 | atomic_inc(&fs_ref->ref); 155 | } 156 | spin_unlock(&dentry->d_lock); 157 | 158 | return priv; 159 | } 160 | 161 | static void fault_fs_ref_put(struct fault_fs_ref *fs_ref) 162 | { 163 | BUG_ON(atomic_read(&fs_ref->ref) == 0); 164 | if (atomic_dec_and_test(&fs_ref->ref)) 165 | wake_up(&fs_ref->wait); 166 | } 167 | 168 | static void fault_fs_ref_wait_grace_period(struct fault_fs_ref *fs_ref) 169 | { 170 | wait_event(fs_ref->wait, atomic_read(&fs_ref->ref) == 0); 171 | } 172 | 173 | static void fault_dentry_remove(struct fault_fs_ref *fs_ref, 174 | struct dentry *dentry) 175 | { 176 | debugfs_remove_recursive(dentry); 177 | fault_fs_ref_wait_grace_period(fs_ref); 178 | } 179 | 180 | static struct fault_group *group_allocate(struct fault_inject *inj, 181 | unsigned int id) 182 | { 183 | int i; 184 | struct fault_group *group; 185 | 186 | group = kzalloc(sizeof(*group), GFP_KERNEL); 187 | if (!group) 188 | return NULL; 189 | 190 | group->inj = inj; 191 | group->id = id; 192 | fault_fs_ref_init(&group->fs_ref); 193 | atomic_set(&group->ref, 0); 194 | INIT_LIST_HEAD(&group->list); 195 | mutex_init(&group->lock); 196 | init_waitqueue_head(&group->wait); 197 | 198 | BUILD_BUG_ON(ARRAY_SIZE(group->faults) != FAULTS_NUM); 199 | BUILD_BUG_ON(ARRAY_SIZE(group->faults) != 200 | ARRAY_SIZE(fault_type_strings)); 201 | for (i = 0; i < ARRAY_SIZE(group->faults); i++) { 202 | group->faults[i] = (struct fault_cfg) { 203 | .type = (enum fault_type)i, 204 | .name = fault_type_strings[i], 205 | .enabled = false, 206 | .task_filter = false, 207 | .hits = ATOMIC64_INIT(0), 208 | .injected = ATOMIC64_INIT(0), 209 | .times = ATOMIC_INIT(-1), 210 | .probability = 100, 211 | .interval = 1, 212 | .delay_us = {0, 0}, 213 | .group = group, 214 | }; 215 | } 216 | 217 | return group; 218 | } 219 | 220 | static void group_free(struct fault_group *group) 221 | { 222 | kfree(group); 223 | } 224 | 225 | static void __group_get(struct fault_group *group) 226 | { 227 | BUG_ON(atomic_read(&group->ref) < 0); 228 | atomic_inc(&group->ref); 229 | } 230 | 231 | /** 232 | * group_get() - Gets group pointer from the fault point. 233 | * 234 | * If fault point has a group pointer, reference will be increased and group 235 | * will be returned. In case of error NULL will be returned. 236 | * 237 | * Function can be called from fault injection, thus any kind of contexts are 238 | * possible, even it can be called from interrupts. 239 | */ 240 | static struct fault_group *group_get(struct fault_inject *inj, 241 | struct fault_point *fault) 242 | { 243 | unsigned long flags; 244 | struct fault_group *group; 245 | 246 | spin_lock_irqsave(&inj->lock, flags); 247 | group = fault->group; 248 | if (likely(group)) { 249 | __group_get(group); 250 | BUG_ON(group->inj != inj); 251 | } 252 | spin_unlock_irqrestore(&inj->lock, flags); 253 | 254 | return group; 255 | } 256 | 257 | static bool group_put(struct fault_group *group) 258 | { 259 | int ref; 260 | 261 | ref = atomic_sub_return(1, &group->ref); 262 | BUG_ON(ref < 0); 263 | 264 | if (ref == 0) { 265 | group_free(group); 266 | return true; 267 | } else if (ref == 1) { 268 | /* 269 | * We do not wake up on 0 reference, since it is stupid to 270 | * wait for group->ref == 0, because an object will be deleted 271 | * underneath waiter. Here we do wake up on last reference. 272 | */ 273 | wake_up(&group->wait); 274 | } 275 | 276 | return false; 277 | } 278 | 279 | static void group_wait_last_ref(struct fault_group *group) 280 | { 281 | wait_event(group->wait, atomic_read(&group->ref) == 1); 282 | } 283 | 284 | static bool group_list_del_and_put(struct fault_group *group) 285 | { 286 | struct fault_group *group2put = NULL; 287 | 288 | spin_lock_irq(&group->inj->lock); 289 | if (likely(!list_empty(&group->list))) { 290 | list_del_init(&group->list); 291 | group2put = group; 292 | } 293 | spin_unlock_irq(&group->inj->lock); 294 | 295 | if (group2put) { 296 | group_put(group2put); 297 | return true; 298 | } else 299 | return false; 300 | } 301 | 302 | static bool __group_unlink_and_put(struct fault_group *group, 303 | struct fault_point *fault) 304 | { 305 | struct fault_group *group2put = NULL; 306 | 307 | spin_lock_irq(&group->inj->lock); 308 | if (fault->group == group) { 309 | group2put = fault->group; 310 | fault->group = NULL; 311 | } 312 | spin_unlock_irq(&group->inj->lock); 313 | 314 | if (group2put) { 315 | group_put(group2put); 316 | return true; 317 | } else 318 | return false; 319 | } 320 | 321 | static bool group_unlink_and_put(struct fault_group *group, 322 | struct fault_point *fault) 323 | { 324 | int i; 325 | bool ok; 326 | 327 | mutex_lock(&group->lock); 328 | ok = __group_unlink_and_put(group, fault); 329 | if (unlikely(!ok)) { 330 | mutex_unlock(&group->lock); 331 | return false; 332 | } 333 | for (i = 0; i < ARRAY_SIZE(group->faults); i++) 334 | if (group->faults[i].enabled) 335 | static_key_slow_dec(&fault->key); 336 | mutex_unlock(&group->lock); 337 | 338 | return true; 339 | } 340 | 341 | static int 342 | group_unlink_from_fault(struct fault_inject *inj, struct fault_point *fault, 343 | struct jump_entry *entry, void *priv) 344 | { 345 | bool ok; 346 | struct fault_group *group = priv; 347 | 348 | if (fault->group == group) { 349 | ok = group_unlink_and_put(group, fault); 350 | BUG_ON(!ok); 351 | } 352 | 353 | return 0; 354 | } 355 | 356 | static void group_list_add_and_get(struct fault_group *group) 357 | { 358 | spin_lock_irq(&group->inj->lock); 359 | list_add(&group->list, &group->inj->list_groups); 360 | __group_get(group); 361 | spin_unlock_irq(&group->inj->lock); 362 | } 363 | 364 | static bool group_link_and_get(struct fault_group *group, 365 | struct fault_point *fault) 366 | { 367 | bool ok = false; 368 | 369 | spin_lock_irq(&group->inj->lock); 370 | if (fault->group == NULL) { 371 | fault->group = group; 372 | __group_get(group); 373 | ok = true; 374 | } 375 | spin_unlock_irq(&group->inj->lock); 376 | 377 | return ok; 378 | } 379 | 380 | static struct fault_group *group_find_and_get(struct fault_inject *inj, 381 | unsigned int id) 382 | { 383 | struct fault_group *group; 384 | 385 | spin_lock_irq(&inj->lock); 386 | list_for_each_entry(group, &inj->list_groups, list) { 387 | if (group->id == id) { 388 | BUG_ON(atomic_read(&group->ref) <= 0); 389 | __group_get(group); 390 | BUG_ON(group->inj != inj); 391 | spin_unlock_irq(&inj->lock); 392 | return group; 393 | } 394 | } 395 | spin_unlock_irq(&inj->lock); 396 | 397 | return NULL; 398 | } 399 | 400 | struct fault_iter { 401 | struct jump_entry *start; 402 | struct jump_entry *iter; 403 | struct jump_entry *stop; 404 | struct fault_group *group; 405 | }; 406 | 407 | static inline struct fault_point *__fault_iter_to_valid( 408 | struct fault_inject *inj, 409 | struct fault_iter *it) 410 | { 411 | for (; it->iter < it->stop; it->iter++) { 412 | struct fault_point *f; 413 | 414 | f = (struct fault_point *)it->iter->key; 415 | if (f->magic != FAULT_MAGIC) 416 | continue; 417 | /* 418 | * See 'jump_label_invalidate_module_init', if we are in 419 | * a module init code - it will be set to zero. Skip it.\ 420 | */ 421 | if (it->iter->code == 0x0) 422 | continue; 423 | /* Warn if somebody is naughty */ 424 | WARN(within_module_init(it->iter->code, inj->mod), 425 | "Fault was injected into module init code, addr 0x%016llx", 426 | it->iter->code); 427 | 428 | /* Check group match if specified */ 429 | if (it->group && it->group != f->group) 430 | continue; 431 | 432 | return f; 433 | } 434 | 435 | return NULL; 436 | } 437 | 438 | static inline struct fault_point *fault_iter_next( 439 | struct fault_inject *inj, 440 | struct fault_iter *it) 441 | { 442 | if (it->iter >= it->stop) 443 | return NULL; 444 | 445 | it->iter++; 446 | 447 | return __fault_iter_to_valid(inj, it); 448 | } 449 | 450 | static inline struct fault_point *fault_iter_curr( 451 | struct fault_inject *inj, 452 | struct fault_iter *it) 453 | { 454 | return __fault_iter_to_valid(inj, it); 455 | } 456 | 457 | static inline bool fault_iter_init(struct fault_inject *inj, 458 | struct fault_iter *it, 459 | struct fault_group *group, 460 | unsigned int pos) 461 | { 462 | if (inj->mod->num_jump_entries && pos >= inj->mod->num_jump_entries) 463 | return false; 464 | 465 | it->start = inj->mod->jump_entries; 466 | it->iter = it->start + pos; 467 | it->stop = it->start + inj->mod->num_jump_entries; 468 | it->group = group; 469 | 470 | return true; 471 | } 472 | 473 | static inline unsigned int fault_iter_pos(struct fault_iter *it) 474 | { 475 | return (it->iter - it->start); 476 | } 477 | 478 | static int for_each_fault_inject_entry(struct fault_inject *inj, 479 | int (*cb)(struct fault_inject *inj, 480 | struct fault_point *fault, 481 | struct jump_entry *entry, 482 | void *priv), 483 | void *priv) 484 | { 485 | struct fault_iter it; 486 | struct fault_point *f; 487 | bool init; 488 | int rc; 489 | 490 | init = fault_iter_init(inj, &it, NULL /* no group */, 0); 491 | BUG_ON(!init); 492 | 493 | for (f = fault_iter_curr(inj, &it); f; f = fault_iter_next(inj, &it)) { 494 | rc = cb(inj, f, it.iter, priv); 495 | if (rc) 496 | return rc; 497 | } 498 | 499 | return 0; 500 | } 501 | 502 | struct fault_as_ret { 503 | struct fault_point *fault; 504 | void *addr; 505 | }; 506 | 507 | static int fault_by_target_addr(struct fault_inject *inj, 508 | struct fault_point *fault, 509 | struct jump_entry *entry, 510 | void *priv) 511 | { 512 | struct fault_as_ret *ret = priv; 513 | 514 | if (entry->target == (unsigned long)ret->addr) { 515 | ret->fault = fault; 516 | return 1; 517 | } 518 | 519 | return 0; 520 | } 521 | 522 | static int fault_by_code_addr(struct fault_inject *inj, 523 | struct fault_point *fault, 524 | struct jump_entry *entry, void *priv) 525 | { 526 | struct fault_as_ret *ret = priv; 527 | 528 | if (entry->code == (unsigned long)ret->addr) { 529 | ret->fault = fault; 530 | return 1; 531 | } 532 | 533 | return 0; 534 | } 535 | 536 | static struct fault_point *find_fault_by_target(struct fault_inject *inj, 537 | void *addr) 538 | { 539 | struct fault_as_ret ret = { 540 | .addr = addr, 541 | .fault = NULL, 542 | }; 543 | 544 | for_each_fault_inject_entry( 545 | inj, fault_by_target_addr, &ret); 546 | 547 | return ret.fault; 548 | } 549 | 550 | static inline void __do_udelay(unsigned int delay_us) 551 | { 552 | unsigned int d; 553 | 554 | while (delay_us) { 555 | d = min(delay_us, 1000u); 556 | udelay(d); 557 | delay_us -= d; 558 | } 559 | } 560 | 561 | static inline void do_fault_delay(struct fault_cfg *cfg) 562 | { 563 | unsigned int beg, end, delay; 564 | 565 | /* Fetch them first to avoid non-atomic set. See @delay_us_store */ 566 | beg = cfg->delay_us.beg; 567 | end = cfg->delay_us.end; 568 | 569 | if (end <= beg) 570 | delay = beg; 571 | else 572 | delay = beg + prandom_u32_max(end - beg + 1); 573 | 574 | __do_udelay(delay); 575 | } 576 | 577 | static inline bool task_may_inject_fault(struct fault_cfg *cfg, 578 | struct task_struct *task) 579 | { 580 | /* This is Linux kernel configuration, see /proc/{pid}/make-it-fail */ 581 | #ifdef CONFIG_FAULT_INJECTION 582 | return !cfg->task_filter || (!in_interrupt() && task->make_it_fail); 583 | #else 584 | return true; 585 | #endif 586 | } 587 | 588 | static int inject_fault(struct fault_cfg *cfg) 589 | { 590 | int ret; 591 | unsigned int bit_off, bit; 592 | unsigned long long hits, injected; 593 | 594 | hits = atomic64_add_return(1, &cfg->hits); 595 | 596 | if (!task_may_inject_fault(cfg, current)) 597 | return 0; 598 | if (atomic_read(&cfg->times) == 0) 599 | return 0; 600 | BUG_ON(cfg->interval == 0); 601 | if ((hits - 1) % cfg->interval) 602 | return 0; 603 | BUG_ON(cfg->probability == 0); 604 | if (cfg->probability < 100 && 605 | cfg->probability <= prandom_u32() % 100) 606 | return 0; 607 | 608 | injected = atomic64_add_return(1, &cfg->injected); 609 | atomic_dec_if_positive(&cfg->times); 610 | 611 | switch (cfg->type) { 612 | case ERROR_FAULT: 613 | /* Except zero bit */ 614 | bit_off = injected % (FAULT_ERRORS_SZ - 1) + 1; 615 | bit = find_next_bit(cfg->error_mask, 616 | FAULT_ERRORS_SZ, bit_off); 617 | /* Start from the beginning */ 618 | if (bit >= FAULT_ERRORS_SZ && bit_off > 1) { 619 | bit = find_next_bit(cfg->error_mask, bit_off, 1); 620 | if (bit >= bit_off) 621 | bit = FAULT_ERRORS_SZ; 622 | } 623 | 624 | if (bit < FAULT_ERRORS_SZ) 625 | ret = -bit; 626 | else 627 | ret = 0; 628 | break; 629 | case DELAY_FAULT: 630 | do_fault_delay(cfg); 631 | ret = 0; 632 | break; 633 | case PANIC_FAULT: 634 | panic("Panic from fault injection"); 635 | ret = 0; 636 | break; 637 | default: 638 | BUG_ON(1); 639 | ret = 0; 640 | break; 641 | } 642 | 643 | return ret; 644 | } 645 | 646 | int inject_faults_by_target(struct fault_inject *inj, void *addr) 647 | { 648 | int i, err = 0; 649 | struct fault_group *group; 650 | struct fault_cfg *cfg; 651 | struct fault_point *fp; 652 | 653 | fp = find_fault_by_target(inj, addr); 654 | BUG_ON(fp == NULL); 655 | 656 | group = group_get(inj, fp); 657 | if (!group) 658 | return 0; 659 | 660 | for (i = 0; i < ARRAY_SIZE(group->faults); i++) { 661 | cfg = &group->faults[i]; 662 | if (!cfg->enabled) 663 | continue; 664 | err = inject_fault(cfg); 665 | if (err) 666 | goto out; 667 | 668 | } 669 | out: 670 | group_put(group); 671 | return err; 672 | } 673 | EXPORT_SYMBOL_GPL(inject_faults_by_target); 674 | 675 | static __printf(2, 3) int __seq_printf(struct seq_file *sf, 676 | const char *fmt, ...) 677 | { 678 | va_list args; 679 | int i; 680 | 681 | va_start(args, fmt); 682 | if (sf) { 683 | i = sf->count; 684 | seq_vprintf(sf, fmt, args); 685 | i = sf->count - i; 686 | } 687 | else 688 | i = vsnprintf(NULL, 0, fmt, args); 689 | va_end(args); 690 | 691 | return i; 692 | } 693 | 694 | static int seq_format_header(struct seq_file *sf, struct fault_inject *inj) 695 | { 696 | unsigned int width; 697 | 698 | BUG_ON(inj->max_func_sz < sizeof(FUNC_HEADER) - 1); 699 | 700 | width = inj->max_func_sz - 701 | (sizeof(FUNC_HEADER) - 1) + 702 | (sizeof(FILE_HEADER) - 1); 703 | 704 | return __seq_printf(sf, "%s %s %s %s %s %*s\n", 705 | FAULT_HEADER, 706 | GROUP_HEADER, 707 | CLASS_HEADER, 708 | ADDR_HEADER, 709 | FUNC_HEADER, 710 | width, 711 | FILE_HEADER); 712 | } 713 | 714 | static int seq_format_fault_point(struct seq_file *sf, 715 | struct fault_inject *inj, 716 | struct fault_point *fault, 717 | struct jump_entry *entry) 718 | { 719 | unsigned long func_sz; 720 | unsigned int file_sz; 721 | const char *file = fault->file; 722 | struct fault_group *group; 723 | /* The following arrays should be in sync with FAULT_LINE_SZ */ 724 | char fault_buf[6] = "-----"; 725 | char group_buf[6] = "-----"; 726 | char class_buf[6] = "-----"; 727 | char addr_buf[19]; 728 | char func_buf[65]; 729 | char file_buf[25]; 730 | char *tmp; 731 | 732 | BUILD_BUG_ON(sizeof(fault_buf) + sizeof(group_buf) + 733 | sizeof(class_buf) + sizeof(addr_buf) + 734 | sizeof(func_buf) + sizeof(file_buf) + 1 735 | != FAULT_LINE_SZ); 736 | 737 | group = group_get(inj, fault); 738 | if (group) { 739 | int i; 740 | 741 | BUILD_BUG_ON(ARRAY_SIZE(fault->group->faults) >= 742 | sizeof(fault_buf)); 743 | /* Fill in fault buffer */ 744 | for (i = 0; i < ARRAY_SIZE(fault->group->faults); i++) { 745 | if (!fault->group->faults[i].enabled) 746 | continue; 747 | fault_buf[i] = fault_type_chars[ 748 | fault->group->faults[i].type]; 749 | } 750 | 751 | /* Fill in group buffer, never overflows */ 752 | snprintf(group_buf, sizeof(group_buf), "%5u", 753 | fault->group->id); 754 | 755 | group_put(group); 756 | group = NULL; 757 | } 758 | 759 | /* Fill in addr buffer, never overflows */ 760 | snprintf(addr_buf, sizeof(addr_buf), "0x%016llx", entry->code); 761 | 762 | /* Fill in class buffer, can overflow */ 763 | if (fault->class) 764 | snprintf(class_buf, sizeof(class_buf), "%5s", fault->class); 765 | 766 | /* Fill in func buffer */ 767 | func_sz = snprintf(func_buf, sizeof(func_buf), "%pF", 768 | (void *)entry->code); 769 | tmp = strchr(func_buf, ' '); 770 | if (tmp) { 771 | /* Skip module name, we really aware where we are */ 772 | func_sz = tmp - func_buf; 773 | func_buf[func_sz] = '\0'; 774 | } 775 | if (func_sz >= sizeof(func_buf)) { 776 | /* Overflow? Mark with dots and do size correction */ 777 | memcpy(func_buf + sizeof(func_buf) - 4, "...", 3); 778 | func_sz = sizeof(func_buf) - 1; 779 | } 780 | 781 | /* Fill in file buffer */ 782 | file = strrchr(fault->file, '/'); 783 | if (file) 784 | file++; 785 | else 786 | file = fault->file; 787 | file_sz = snprintf(file_buf, sizeof(file_buf), "%s:%u", 788 | file, fault->line); 789 | if (file_sz >= sizeof(file_buf)) { 790 | /* Overflow? Mark with dots and do size correction */ 791 | memcpy(file_buf + sizeof(file_buf) - 4, "...", 3); 792 | file_sz = sizeof(file_buf) - 1; 793 | } 794 | 795 | /* Update max */ 796 | if (inj->max_func_sz < func_sz) 797 | inj->max_func_sz = func_sz; 798 | 799 | return __seq_printf(sf, "%s %s %s %s %s %*s\n", 800 | fault_buf, 801 | group_buf, 802 | class_buf, 803 | addr_buf, 804 | func_buf, 805 | (unsigned int)(inj->max_func_sz - func_sz + file_sz), 806 | file_buf); 807 | } 808 | 809 | static int dry_format_fault_point(struct fault_inject *inj, 810 | struct fault_point *fault, 811 | struct jump_entry *entry, void *notused) 812 | { 813 | int rc; 814 | 815 | rc = seq_format_fault_point(NULL, inj, fault, entry); 816 | if (unlikely(rc < 0)) 817 | return rc; 818 | return 0; 819 | } 820 | 821 | 822 | struct fault_line { 823 | void *priv; 824 | struct mutex mutex; 825 | unsigned int loff; 826 | unsigned int len; 827 | char buf[FAULT_LINE_SZ]; 828 | }; 829 | 830 | static inline struct fault_line *fault_line_create(void *priv) 831 | { 832 | struct fault_line *line; 833 | 834 | line = kmalloc(sizeof(*line), GFP_KERNEL); 835 | if (unlikely(line == NULL)) 836 | return NULL; 837 | 838 | mutex_init(&line->mutex); 839 | line->priv = priv; 840 | line->loff = 0; 841 | line->len = 0; 842 | 843 | return line; 844 | } 845 | 846 | static ssize_t cache_user_line(struct fault_line *line, 847 | const char __user *from, 848 | size_t *off, size_t size) 849 | { 850 | unsigned int len, res, loff; 851 | char *end; 852 | 853 | BUG_ON(*off >= size); 854 | BUG_ON(line->loff >= sizeof(line->buf)); 855 | len = min(size - *off, sizeof(line->buf) - line->loff); 856 | res = copy_from_user(line->buf + line->loff, from + *off, len); 857 | if (res) { 858 | line->len = 0; 859 | line->loff = 0; 860 | return -EFAULT; 861 | } 862 | 863 | loff = line->loff; 864 | line->loff += len; 865 | end = strnstr(line->buf, "\n", line->loff); 866 | if (!end) { 867 | if (line->loff == sizeof(line->buf)) { 868 | line->len = 0; 869 | line->loff = 0; 870 | /* Buffer is over but where is \n? */ 871 | return -EINVAL; 872 | } 873 | return *off + len; 874 | } 875 | end[0] = '\0'; 876 | line->len = end - line->buf; 877 | line->loff = 0; 878 | /* Advance the offset to the beginning of a new line */ 879 | *off += line->len - loff + 1; 880 | 881 | return 0; 882 | } 883 | 884 | /** 885 | * The end of common fault code, a lot of debugfs stuff is the following. 886 | */ 887 | 888 | #define FAULT_SIMPLE_ATTRIBUTE(__fops, __get, __set, __fmt, __conv) \ 889 | static int __fops ## _open(struct inode *inode, struct file *file) \ 890 | { \ 891 | void *priv; \ 892 | \ 893 | __simple_attr_check_format(__fmt, 0ull); \ 894 | priv = fault_fs_ref_get(file->f_path.dentry, __conv); \ 895 | if (unlikely(!priv)) \ 896 | return -ENOENT; \ 897 | return simple_attr_open(inode, file, __get, __set, __fmt); \ 898 | } \ 899 | static int __fops ## _close(struct inode *inode, struct file *file) \ 900 | { \ 901 | simple_attr_release(inode, file); \ 902 | fault_fs_ref_put(__conv(priv_from_dentry(file->f_path.dentry)));\ 903 | return 0; \ 904 | } \ 905 | static const struct file_operations __fops = { \ 906 | .owner = THIS_MODULE, \ 907 | .open = __fops ## _open, \ 908 | .release = __fops ## _close, \ 909 | .read = simple_attr_read, \ 910 | .write = simple_attr_write, \ 911 | .llseek = generic_file_llseek, \ 912 | } 913 | 914 | static int enable_read_op(void *data, u64 *val) 915 | { 916 | struct fault_cfg *cfg = data; 917 | 918 | *val = cfg->enabled; 919 | 920 | return 0; 921 | } 922 | 923 | struct fault_point_enabled { 924 | struct fault_group *group; 925 | bool to_enable; 926 | }; 927 | 928 | static int __fault_point_enable(struct fault_inject *inj, 929 | struct fault_point *fault, 930 | struct jump_entry *entry, void *priv) 931 | { 932 | struct fault_point_enabled *e = priv; 933 | 934 | if (e->group == fault->group) { 935 | if (e->to_enable) 936 | static_key_slow_inc(&fault->key); 937 | else 938 | static_key_slow_dec(&fault->key); 939 | } 940 | 941 | return 0; 942 | } 943 | 944 | static int enable_write_op(void *data, u64 val) 945 | { 946 | struct fault_cfg *cfg = data; 947 | struct fault_point_enabled e; 948 | 949 | e.group = cfg->group; 950 | e.to_enable = !!val; 951 | 952 | mutex_lock(&cfg->group->lock); 953 | if (likely(e.to_enable ^ cfg->enabled)) { 954 | for_each_fault_inject_entry( 955 | e.group->inj, __fault_point_enable, &e); 956 | cfg->enabled = e.to_enable; 957 | } 958 | mutex_unlock(&cfg->group->lock); 959 | 960 | return 0; 961 | } 962 | FAULT_SIMPLE_ATTRIBUTE(enable_fault_fops, enable_read_op, 963 | enable_write_op, "%llu\n", 964 | ref_from_fault_cfg); 965 | 966 | static int hits_read_op(void *data, u64 *val) 967 | { 968 | struct fault_cfg *cfg = data; 969 | 970 | *val = atomic64_read(&cfg->hits); 971 | 972 | return 0; 973 | } 974 | FAULT_SIMPLE_ATTRIBUTE(hits_fault_fops, hits_read_op, 975 | NULL, "%llu\n", 976 | ref_from_fault_cfg); 977 | 978 | static int injected_read_op(void *data, u64 *val) 979 | { 980 | struct fault_cfg *cfg = data; 981 | 982 | *val = atomic64_read(&cfg->injected); 983 | 984 | return 0; 985 | } 986 | FAULT_SIMPLE_ATTRIBUTE(injected_fault_fops, injected_read_op, 987 | NULL, "%llu\n", 988 | ref_from_fault_cfg); 989 | 990 | static int times_read_op(void *data, u64 *val) 991 | { 992 | struct fault_cfg *cfg = data; 993 | 994 | *val = atomic_read(&cfg->times); 995 | 996 | return 0; 997 | } 998 | 999 | static int times_write_op(void *data, u64 val) 1000 | { 1001 | struct fault_cfg *cfg = data; 1002 | 1003 | if (val == 0) 1004 | return -EINVAL; 1005 | if ((int64_t)val < 0) 1006 | val = -1; 1007 | 1008 | atomic_set(&cfg->times, val); 1009 | 1010 | return 0; 1011 | } 1012 | FAULT_SIMPLE_ATTRIBUTE(times_fault_fops, times_read_op, 1013 | times_write_op, "%lld\n", 1014 | ref_from_fault_cfg); 1015 | 1016 | static int probability_read_op(void *data, u64 *val) 1017 | { 1018 | struct fault_cfg *cfg = data; 1019 | 1020 | *val = cfg->probability; 1021 | 1022 | return 0; 1023 | } 1024 | 1025 | static int probability_write_op(void *data, u64 val) 1026 | { 1027 | struct fault_cfg *cfg = data; 1028 | 1029 | if (val == 0 || val > 100) 1030 | return -EINVAL; 1031 | 1032 | cfg->probability = val; 1033 | 1034 | return 0; 1035 | } 1036 | FAULT_SIMPLE_ATTRIBUTE(probability_fault_fops, probability_read_op, 1037 | probability_write_op, "%llu\n", 1038 | ref_from_fault_cfg); 1039 | 1040 | static int interval_read_op(void *data, u64 *val) 1041 | { 1042 | struct fault_cfg *cfg = data; 1043 | 1044 | *val = cfg->interval; 1045 | 1046 | return 0; 1047 | } 1048 | 1049 | static int interval_write_op(void *data, u64 val) 1050 | { 1051 | struct fault_cfg *cfg = data; 1052 | 1053 | if (val == 0) 1054 | return -EINVAL; 1055 | 1056 | cfg->interval = val; 1057 | 1058 | return 0; 1059 | } 1060 | FAULT_SIMPLE_ATTRIBUTE(interval_fault_fops, interval_read_op, 1061 | interval_write_op, "%llu\n", 1062 | ref_from_fault_cfg); 1063 | 1064 | /* This is Linux kernel configuration, see /proc/{pid}/make-it-fail */ 1065 | #ifdef CONFIG_FAULT_INJECTION 1066 | 1067 | static int task_filter_read_op(void *data, u64 *val) 1068 | { 1069 | struct fault_cfg *cfg = data; 1070 | 1071 | *val = cfg->task_filter; 1072 | 1073 | return 0; 1074 | } 1075 | 1076 | static int task_filter_write_op(void *data, u64 val) 1077 | { 1078 | struct fault_cfg *cfg = data; 1079 | 1080 | cfg->task_filter = !!val; 1081 | 1082 | return 0; 1083 | } 1084 | FAULT_SIMPLE_ATTRIBUTE(task_filter_fault_fops, task_filter_read_op, 1085 | task_filter_write_op, "%llu\n", 1086 | ref_from_fault_cfg); 1087 | 1088 | #endif /* CONFIG_FAULT_INJECTION */ 1089 | 1090 | /** 1091 | * strerror() - convert errno values to string 1092 | * @err: errno value [input] 1093 | * 1094 | * The function converts errno values to strings. 1095 | * 1096 | * Unknown errno values are converted to their decimal 1097 | * string representation. 1098 | * 1099 | * The function is not multithread safe as it places the 1100 | * conversion of unknown errno values in a static variable. 1101 | */ 1102 | static const char *strerror(long err) 1103 | { 1104 | /* decimal digits needed <= (2.5 * sizeof(err)) */ 1105 | static char buf[(sizeof(err) << 1) + sizeof(err)]; 1106 | 1107 | if (unlikely(err >= 0)) { 1108 | sprintf(buf, "%ld", err); 1109 | return buf; 1110 | } 1111 | switch (err) { 1112 | #define CASE2STR(x) case (x): return #x 1113 | CASE2STR(-EPERM); 1114 | CASE2STR(-ENOENT); 1115 | CASE2STR(-ESRCH); 1116 | CASE2STR(-EINTR); 1117 | CASE2STR(-EIO); 1118 | CASE2STR(-ENXIO); 1119 | CASE2STR(-E2BIG); 1120 | CASE2STR(-ENOEXEC); 1121 | CASE2STR(-EBADF); 1122 | CASE2STR(-ECHILD); 1123 | CASE2STR(-EAGAIN); 1124 | CASE2STR(-ENOMEM); 1125 | CASE2STR(-EACCES); 1126 | CASE2STR(-EFAULT); 1127 | CASE2STR(-ENOTBLK); 1128 | CASE2STR(-EBUSY); 1129 | CASE2STR(-EEXIST); 1130 | CASE2STR(-EXDEV); 1131 | CASE2STR(-ENODEV); 1132 | CASE2STR(-ENOTDIR); 1133 | CASE2STR(-EISDIR); 1134 | CASE2STR(-EINVAL); 1135 | CASE2STR(-ENFILE); 1136 | CASE2STR(-EMFILE); 1137 | CASE2STR(-ENOTTY); 1138 | CASE2STR(-ETXTBSY); 1139 | CASE2STR(-EFBIG); 1140 | CASE2STR(-ENOSPC); 1141 | CASE2STR(-ESPIPE); 1142 | CASE2STR(-EROFS); 1143 | CASE2STR(-EMLINK); 1144 | CASE2STR(-EPIPE); 1145 | CASE2STR(-EDOM); 1146 | CASE2STR(-ERANGE); 1147 | CASE2STR(-EDEADLK); 1148 | CASE2STR(-ENAMETOOLONG); 1149 | CASE2STR(-ENOLCK); 1150 | CASE2STR(-ENOSYS); 1151 | CASE2STR(-ENOTEMPTY); 1152 | CASE2STR(-ELOOP); 1153 | CASE2STR(-ENOMSG); 1154 | CASE2STR(-EIDRM); 1155 | CASE2STR(-ECHRNG); 1156 | CASE2STR(-EL2NSYNC); 1157 | CASE2STR(-EL3HLT); 1158 | CASE2STR(-EL3RST); 1159 | CASE2STR(-ELNRNG); 1160 | CASE2STR(-EUNATCH); 1161 | CASE2STR(-ENOCSI); 1162 | CASE2STR(-EL2HLT); 1163 | CASE2STR(-EBADE); 1164 | CASE2STR(-EBADR); 1165 | CASE2STR(-EXFULL); 1166 | CASE2STR(-ENOANO); 1167 | CASE2STR(-EBADRQC); 1168 | CASE2STR(-EBADSLT); 1169 | CASE2STR(-EBFONT); 1170 | CASE2STR(-ENOSTR); 1171 | CASE2STR(-ENODATA); 1172 | CASE2STR(-ETIME); 1173 | CASE2STR(-ENOSR); 1174 | CASE2STR(-ENONET); 1175 | CASE2STR(-ENOPKG); 1176 | CASE2STR(-EREMOTE); 1177 | CASE2STR(-ENOLINK); 1178 | CASE2STR(-EADV); 1179 | CASE2STR(-ESRMNT); 1180 | CASE2STR(-ECOMM); 1181 | CASE2STR(-EPROTO); 1182 | CASE2STR(-EMULTIHOP); 1183 | CASE2STR(-EDOTDOT); 1184 | CASE2STR(-EBADMSG); 1185 | CASE2STR(-EOVERFLOW); 1186 | CASE2STR(-ENOTUNIQ); 1187 | CASE2STR(-EBADFD); 1188 | CASE2STR(-EREMCHG); 1189 | CASE2STR(-ELIBACC); 1190 | CASE2STR(-ELIBBAD); 1191 | CASE2STR(-ELIBSCN); 1192 | CASE2STR(-ELIBMAX); 1193 | CASE2STR(-ELIBEXEC); 1194 | CASE2STR(-EILSEQ); 1195 | CASE2STR(-ERESTART); 1196 | CASE2STR(-ESTRPIPE); 1197 | CASE2STR(-EUSERS); 1198 | CASE2STR(-ENOTSOCK); 1199 | CASE2STR(-EDESTADDRREQ); 1200 | CASE2STR(-EMSGSIZE); 1201 | CASE2STR(-EPROTOTYPE); 1202 | CASE2STR(-ENOPROTOOPT); 1203 | CASE2STR(-EPROTONOSUPPORT); 1204 | CASE2STR(-ESOCKTNOSUPPORT); 1205 | CASE2STR(-EOPNOTSUPP); 1206 | CASE2STR(-EPFNOSUPPORT); 1207 | CASE2STR(-EAFNOSUPPORT); 1208 | CASE2STR(-EADDRINUSE); 1209 | CASE2STR(-EADDRNOTAVAIL); 1210 | CASE2STR(-ENETDOWN); 1211 | CASE2STR(-ENETUNREACH); 1212 | CASE2STR(-ENETRESET); 1213 | CASE2STR(-ECONNABORTED); 1214 | CASE2STR(-ECONNRESET); 1215 | CASE2STR(-ENOBUFS); 1216 | CASE2STR(-EISCONN); 1217 | CASE2STR(-ENOTCONN); 1218 | CASE2STR(-ESHUTDOWN); 1219 | CASE2STR(-ETOOMANYREFS); 1220 | CASE2STR(-ETIMEDOUT); 1221 | CASE2STR(-ECONNREFUSED); 1222 | CASE2STR(-EHOSTDOWN); 1223 | CASE2STR(-EHOSTUNREACH); 1224 | CASE2STR(-EALREADY); 1225 | CASE2STR(-EINPROGRESS); 1226 | CASE2STR(-ESTALE); 1227 | CASE2STR(-EUCLEAN); 1228 | CASE2STR(-ENOTNAM); 1229 | CASE2STR(-ENAVAIL); 1230 | CASE2STR(-EISNAM); 1231 | CASE2STR(-EREMOTEIO); 1232 | CASE2STR(-EDQUOT); 1233 | CASE2STR(-ENOMEDIUM); 1234 | CASE2STR(-EMEDIUMTYPE); 1235 | CASE2STR(-ECANCELED); 1236 | CASE2STR(-ENOKEY); 1237 | CASE2STR(-EKEYEXPIRED); 1238 | CASE2STR(-EKEYREVOKED); 1239 | CASE2STR(-EKEYREJECTED); 1240 | CASE2STR(-EOWNERDEAD); 1241 | CASE2STR(-ENOTRECOVERABLE); 1242 | CASE2STR(-ERFKILL); 1243 | CASE2STR(-EHWPOISON); 1244 | CASE2STR(-ERESTARTSYS); 1245 | CASE2STR(-ERESTARTNOINTR); 1246 | CASE2STR(-ERESTARTNOHAND); 1247 | CASE2STR(-ENOIOCTLCMD); 1248 | CASE2STR(-ERESTART_RESTARTBLOCK); 1249 | CASE2STR(-EBADHANDLE); 1250 | CASE2STR(-ENOTSYNC); 1251 | CASE2STR(-EBADCOOKIE); 1252 | CASE2STR(-ENOTSUPP); 1253 | CASE2STR(-ETOOSMALL); 1254 | CASE2STR(-ESERVERFAULT); 1255 | CASE2STR(-EBADTYPE); 1256 | CASE2STR(-EJUKEBOX); 1257 | CASE2STR(-EIOCBQUEUED); 1258 | default: 1259 | break; 1260 | } 1261 | sprintf(buf, "%ld", err); 1262 | return buf; 1263 | } 1264 | 1265 | /** 1266 | * str2error() - convert string to errno 1267 | * @err_str: string value [input] 1268 | * 1269 | * The function converts string to errno. 1270 | * 1271 | * In case of unknown string 0 will be returned. 1272 | * 1273 | * This function does not care about performance and 1274 | * simply executes 'strerror' for each errno 1275 | * in range [-255, 0) and does strstr. 1276 | */ 1277 | static long str2error(const char *err_str) 1278 | { 1279 | long err; 1280 | const char *not_an_err; 1281 | char first; 1282 | 1283 | if (unlikely(err_str == NULL)) 1284 | return 0; 1285 | 1286 | first = err_str[0]; 1287 | if (unlikely(first != '-' && first != 'E')) 1288 | return 0; 1289 | 1290 | not_an_err = strerror(0) + (first == 'E'); 1291 | 1292 | for (err = -255; err < 0; err++) { 1293 | const char *e = strerror(err) + (first == 'E'); 1294 | 1295 | if (e == not_an_err) 1296 | continue; 1297 | if (!strcmp(e, err_str)) 1298 | return err; 1299 | } 1300 | 1301 | return 0; 1302 | } 1303 | 1304 | static void *__errors_fault_find_next(struct fault_cfg *cfg, loff_t *pos) 1305 | { 1306 | unsigned long bit; 1307 | 1308 | if (*pos >= FAULT_ERRORS_SZ) 1309 | return NULL; 1310 | 1311 | bit = find_next_bit(cfg->error_mask, FAULT_ERRORS_SZ, *pos); 1312 | /* 0 bit means succeess, can't be set */ 1313 | BUG_ON(bit == 0); 1314 | if (bit >= FAULT_ERRORS_SZ) 1315 | /* The end, no error */ 1316 | return NULL; 1317 | 1318 | *pos = bit; 1319 | 1320 | return (void *)bit; 1321 | } 1322 | 1323 | static void *errors_fault_seq_start(struct seq_file *sf, loff_t *pos) 1324 | { 1325 | struct fault_line *line = sf->private; 1326 | struct fault_cfg *cfg = line->priv; 1327 | 1328 | return __errors_fault_find_next(cfg, pos); 1329 | } 1330 | 1331 | static int errors_fault_seq_show(struct seq_file *sf, void *vp) 1332 | { 1333 | unsigned long bit = (unsigned long)vp; 1334 | 1335 | __seq_printf(sf, "%s", strerror(-(long)bit)); 1336 | 1337 | return 0; 1338 | } 1339 | 1340 | static void *errors_fault_seq_next(struct seq_file *sf, void *vp, loff_t *pos) 1341 | { 1342 | struct fault_line *line = sf->private; 1343 | struct fault_cfg *cfg = line->priv; 1344 | void *ret; 1345 | 1346 | ++*pos; 1347 | ret = __errors_fault_find_next(cfg, pos); 1348 | if (ret) 1349 | __seq_printf(sf, ","); 1350 | else 1351 | __seq_printf(sf, "\n"); 1352 | 1353 | return ret; 1354 | } 1355 | 1356 | static void errors_fault_seq_stop(struct seq_file *sf, void *vp) 1357 | { 1358 | } 1359 | 1360 | static const struct seq_operations errors_fault_seq_ops = { 1361 | .start = errors_fault_seq_start, 1362 | .show = errors_fault_seq_show, 1363 | .next = errors_fault_seq_next, 1364 | .stop = errors_fault_seq_stop, 1365 | }; 1366 | 1367 | static int errors_fault_open(struct inode *inode, struct file *file) 1368 | { 1369 | struct dentry *dentry = file->f_path.dentry; 1370 | struct fault_cfg *cfg; 1371 | struct fault_line *line; 1372 | struct seq_file *sf; 1373 | int rc; 1374 | 1375 | cfg = fault_fs_ref_get(dentry, ref_from_fault_cfg); 1376 | if (unlikely(!cfg)) 1377 | return -ENOENT; 1378 | 1379 | rc = seq_open(file, &errors_fault_seq_ops); 1380 | if (unlikely(rc)) { 1381 | fault_fs_ref_put(&cfg->group->fs_ref); 1382 | return rc; 1383 | } 1384 | 1385 | line = fault_line_create(cfg); 1386 | if (unlikely(line == NULL)) { 1387 | seq_release(inode, file); 1388 | fault_fs_ref_put(&cfg->group->fs_ref); 1389 | return -ENOMEM; 1390 | } 1391 | 1392 | sf = file->private_data; 1393 | sf->private = line; 1394 | 1395 | return nonseekable_open(inode, file); 1396 | } 1397 | 1398 | static int errors_fault_close(struct inode *inode, struct file *file) 1399 | { 1400 | struct seq_file *sf = file->private_data; 1401 | struct fault_line *line = sf->private; 1402 | struct fault_cfg *cfg = line->priv; 1403 | struct fault_group *group = cfg->group; 1404 | 1405 | seq_release(inode, file); 1406 | fault_fs_ref_put(&group->fs_ref); 1407 | kfree(line); 1408 | 1409 | return 0; 1410 | } 1411 | 1412 | static ssize_t __errors_store(struct fault_line *line) 1413 | { 1414 | const char *beg = line->buf, *end; 1415 | unsigned int len; 1416 | char buf[16]; 1417 | long err; 1418 | struct fault_cfg *cfg = line->priv; 1419 | bool done = false; 1420 | DECLARE_BITMAP(error_mask, FAULT_ERRORS_SZ); 1421 | 1422 | bitmap_zero(error_mask, FAULT_ERRORS_SZ); 1423 | while (!done) { 1424 | end = strchr(beg, ','); 1425 | if (!end) { 1426 | done = true; 1427 | end = line->buf + line->len; 1428 | if (beg == end) 1429 | break; 1430 | } 1431 | len = end - beg; 1432 | if (len > sizeof(buf) - 1) 1433 | return -EINVAL; 1434 | memcpy(buf, beg, len); 1435 | buf[len] = '\0'; 1436 | err = str2error(buf); 1437 | if (err == 0) 1438 | return -EINVAL; 1439 | if (err >= FAULT_ERRORS_SZ) 1440 | return -EINVAL; 1441 | 1442 | set_bit(abs(err), error_mask); 1443 | 1444 | beg = end + 1; 1445 | } 1446 | 1447 | /* Do not care about concurrent access */ 1448 | bitmap_copy(cfg->error_mask, error_mask, FAULT_ERRORS_SZ); 1449 | 1450 | return 0; 1451 | } 1452 | 1453 | static ssize_t errors_fault_write(struct file *file, 1454 | const char __user *u_buf, 1455 | size_t u_sz, loff_t *pos) 1456 | { 1457 | struct seq_file *sf = file->private_data; 1458 | struct fault_line *line = sf->private; 1459 | size_t u_off = 0; 1460 | int rc; 1461 | 1462 | if (u_sz == 0) 1463 | return 0; 1464 | 1465 | rc = mutex_lock_interruptible(&line->mutex); 1466 | if (unlikely(rc)) 1467 | return rc; 1468 | rc = cache_user_line(line, u_buf, &u_off, u_sz); 1469 | if (rc) 1470 | goto out; 1471 | 1472 | rc = __errors_store(line); 1473 | out: 1474 | mutex_unlock(&line->mutex); 1475 | 1476 | return rc ?: u_sz; 1477 | } 1478 | 1479 | static const struct file_operations errors_fault_fops = { 1480 | .owner = THIS_MODULE, 1481 | .open = errors_fault_open, 1482 | .release = errors_fault_close, 1483 | .read = seq_read, 1484 | .write = errors_fault_write, 1485 | .llseek = no_llseek, 1486 | }; 1487 | 1488 | static void *delay_us_fault_seq_start(struct seq_file *sf, loff_t *pos) 1489 | { 1490 | return *pos ? NULL : sf->private; 1491 | } 1492 | 1493 | static int delay_us_fault_seq_show(struct seq_file *sf, void *vp) 1494 | { 1495 | struct fault_line *line = vp; 1496 | struct fault_cfg *cfg = line->priv; 1497 | 1498 | __seq_printf(sf, "%u:%u\n", 1499 | cfg->delay_us.beg, 1500 | cfg->delay_us.end); 1501 | 1502 | return 0; 1503 | } 1504 | 1505 | static void *delay_us_fault_seq_next(struct seq_file *sf, void *vp, loff_t *pos) 1506 | { 1507 | ++*pos; 1508 | return NULL; 1509 | } 1510 | 1511 | static void delay_us_fault_seq_stop(struct seq_file *sf, void *vp) 1512 | { 1513 | } 1514 | 1515 | static const struct seq_operations delay_us_fault_seq_ops = { 1516 | .start = delay_us_fault_seq_start, 1517 | .show = delay_us_fault_seq_show, 1518 | .next = delay_us_fault_seq_next, 1519 | .stop = delay_us_fault_seq_stop, 1520 | }; 1521 | 1522 | static int delay_us_fault_open(struct inode *inode, struct file *file) 1523 | { 1524 | struct dentry *dentry = file->f_path.dentry; 1525 | struct fault_cfg *cfg; 1526 | struct fault_line *line; 1527 | struct seq_file *sf; 1528 | int rc; 1529 | 1530 | cfg = fault_fs_ref_get(dentry, ref_from_fault_cfg); 1531 | if (unlikely(!cfg)) 1532 | return -ENOENT; 1533 | 1534 | rc = seq_open(file, &delay_us_fault_seq_ops); 1535 | if (unlikely(rc)) { 1536 | fault_fs_ref_put(&cfg->group->fs_ref); 1537 | return rc; 1538 | } 1539 | 1540 | line = fault_line_create(cfg); 1541 | if (unlikely(line == NULL)) { 1542 | seq_release(inode, file); 1543 | fault_fs_ref_put(&cfg->group->fs_ref); 1544 | return -ENOMEM; 1545 | } 1546 | 1547 | sf = file->private_data; 1548 | sf->private = line; 1549 | 1550 | return nonseekable_open(inode, file); 1551 | } 1552 | 1553 | static int delay_us_fault_close(struct inode *inode, struct file *file) 1554 | { 1555 | struct seq_file *sf = file->private_data; 1556 | struct fault_line *line = sf->private; 1557 | struct fault_cfg *cfg = line->priv; 1558 | struct fault_group *group = cfg->group; 1559 | 1560 | seq_release(inode, file); 1561 | fault_fs_ref_put(&group->fs_ref); 1562 | kfree(line); 1563 | 1564 | return 0; 1565 | } 1566 | 1567 | /** 1568 | * __delay_us_store() - Parses delay_us input. 1569 | * 1570 | * Function accepts '%u:%u' range of delays or single '%u' value. 1571 | * The idea is to do random us delay in some specified range. 1572 | * If single value is passed then exactly this value will be used 1573 | * for introducing delays. 1574 | * 1575 | * -EINVAL will be returned if parsing failed or the range is incorrect. 1576 | */ 1577 | static ssize_t __delay_us_store(struct fault_line *line) 1578 | { 1579 | int rc; 1580 | unsigned int beg, end; 1581 | struct fault_cfg *cfg = line->priv; 1582 | 1583 | rc = sscanf(line->buf, "%u:%u", &beg, &end); 1584 | if (rc < 1) 1585 | return -EINVAL; 1586 | else if (rc == 1) 1587 | end = beg; 1588 | if (end < beg) 1589 | return -EINVAL; 1590 | 1591 | cfg->delay_us.beg = beg; 1592 | cfg->delay_us.end = end; 1593 | 1594 | return 0; 1595 | } 1596 | 1597 | static ssize_t delay_us_fault_write(struct file *file, 1598 | const char __user *u_buf, 1599 | size_t u_sz, loff_t *pos) 1600 | { 1601 | struct seq_file *sf = file->private_data; 1602 | struct fault_line *line = sf->private; 1603 | size_t u_off = 0; 1604 | int rc; 1605 | 1606 | if (u_sz == 0) 1607 | return 0; 1608 | 1609 | rc = mutex_lock_interruptible(&line->mutex); 1610 | if (unlikely(rc)) 1611 | return rc; 1612 | rc = cache_user_line(line, u_buf, &u_off, u_sz); 1613 | if (rc) 1614 | goto out; 1615 | 1616 | rc = __delay_us_store(line); 1617 | out: 1618 | mutex_unlock(&line->mutex); 1619 | 1620 | return rc ?: u_sz; 1621 | } 1622 | 1623 | static const struct file_operations delay_us_fault_fops = { 1624 | .owner = THIS_MODULE, 1625 | .open = delay_us_fault_open, 1626 | .release = delay_us_fault_close, 1627 | .read = seq_read, 1628 | .write = delay_us_fault_write, 1629 | .llseek = no_llseek, 1630 | }; 1631 | 1632 | static int group_add_fault_point(struct fault_group *group, void *addr) 1633 | { 1634 | int i; 1635 | bool ok; 1636 | struct fault_as_ret ret = { 1637 | .addr = addr, 1638 | .fault = NULL, 1639 | }; 1640 | 1641 | for_each_fault_inject_entry( 1642 | group->inj, fault_by_code_addr, &ret); 1643 | if (unlikely(ret.fault == NULL)) 1644 | return -EINVAL; 1645 | 1646 | mutex_lock(&group->lock); 1647 | ok = group_link_and_get(group, ret.fault); 1648 | if (unlikely(!ok)) { 1649 | mutex_unlock(&group->lock); 1650 | return -EINVAL; 1651 | } 1652 | BUG_ON(static_key_enabled(&ret.fault->key)); 1653 | for (i = 0; i < ARRAY_SIZE(group->faults); i++) 1654 | if (group->faults[i].enabled) 1655 | static_key_slow_inc(&ret.fault->key); 1656 | mutex_unlock(&group->lock); 1657 | 1658 | return 0; 1659 | } 1660 | 1661 | static int group_del_fault_point(struct fault_group *group, void *addr) 1662 | { 1663 | bool ok; 1664 | struct fault_as_ret ret = { 1665 | .addr = addr, 1666 | .fault = NULL, 1667 | }; 1668 | 1669 | for_each_fault_inject_entry( 1670 | group->inj, fault_by_code_addr, &ret); 1671 | if (unlikely(ret.fault == NULL)) 1672 | return -EINVAL; 1673 | ok = group_unlink_and_put(group, ret.fault); 1674 | if (unlikely(!ok)) 1675 | return -ENOENT; 1676 | 1677 | return 0; 1678 | } 1679 | 1680 | static ssize_t group_parse_address_for_each(struct fault_line *line, 1681 | const char __user *u_buf, 1682 | size_t u_sz, 1683 | int (*cb)(struct fault_group *, 1684 | void *)) 1685 | { 1686 | ssize_t rc; 1687 | unsigned long long addr; 1688 | size_t u_off = 0; 1689 | const char *str; 1690 | 1691 | do { 1692 | rc = cache_user_line(line, u_buf, &u_off, u_sz); 1693 | if (rc) 1694 | return rc; 1695 | 1696 | str = strnstr(line->buf, "0x", line->len); 1697 | if (likely(str)) { 1698 | rc = sscanf(str, "0x%llx", &addr); 1699 | if (unlikely(rc != 1)) 1700 | return -EINVAL; 1701 | 1702 | rc = cb(line->priv, (void *)addr); 1703 | if (unlikely(rc)) 1704 | return -EINVAL; 1705 | } 1706 | 1707 | } while (u_off < u_sz); 1708 | 1709 | return u_sz; 1710 | } 1711 | 1712 | static int generic_wr_fault_points_open(struct inode *inode, struct file *file) 1713 | { 1714 | struct dentry *dentry = file->f_path.dentry; 1715 | struct fault_group *group; 1716 | struct fault_line *line; 1717 | 1718 | group = fault_fs_ref_get(dentry, ref_from_fault_group); 1719 | if (unlikely(!group)) 1720 | return -ENOENT; 1721 | 1722 | line = fault_line_create(group); 1723 | if (unlikely(line == NULL)) { 1724 | fault_fs_ref_put(&group->fs_ref); 1725 | return -ENOMEM; 1726 | } 1727 | 1728 | file->private_data = line; 1729 | 1730 | return nonseekable_open(inode, file); 1731 | } 1732 | 1733 | static int generic_wr_fault_points_close(struct inode *inode, struct file *file) 1734 | { 1735 | struct fault_line *line = file->private_data; 1736 | struct fault_group *group = line->priv; 1737 | 1738 | fault_fs_ref_put(&group->fs_ref); 1739 | kfree(line); 1740 | 1741 | return 0; 1742 | } 1743 | 1744 | static ssize_t add_fault_points_write(struct file *file, 1745 | const char __user *u_buf, 1746 | size_t u_sz, loff_t *pos) 1747 | { 1748 | struct fault_line *line = file->private_data; 1749 | int rc; 1750 | 1751 | if (u_sz == 0) 1752 | return 0; 1753 | 1754 | rc = mutex_lock_interruptible(&line->mutex); 1755 | if (unlikely(rc)) 1756 | return rc; 1757 | rc = group_parse_address_for_each(line, u_buf, u_sz, 1758 | group_add_fault_point); 1759 | mutex_unlock(&line->mutex); 1760 | 1761 | return rc; 1762 | } 1763 | 1764 | static const struct file_operations add_fault_points_fops = { 1765 | .owner = THIS_MODULE, 1766 | .open = generic_wr_fault_points_open, 1767 | .release = generic_wr_fault_points_close, 1768 | .write = add_fault_points_write, 1769 | .llseek = no_llseek, 1770 | }; 1771 | 1772 | static ssize_t del_fault_points_write(struct file *file, 1773 | const char __user *u_buf, 1774 | size_t u_sz, loff_t *pos) 1775 | { 1776 | struct fault_line *line = file->private_data; 1777 | int rc; 1778 | 1779 | if (u_sz == 0) 1780 | return 0; 1781 | 1782 | rc = mutex_lock_interruptible(&line->mutex); 1783 | if (unlikely(rc)) 1784 | return rc; 1785 | rc = group_parse_address_for_each(line, u_buf, u_sz, 1786 | group_del_fault_point); 1787 | mutex_unlock(&line->mutex); 1788 | 1789 | return rc; 1790 | } 1791 | 1792 | static const struct file_operations del_fault_points_fops = { 1793 | .owner = THIS_MODULE, 1794 | .open = generic_wr_fault_points_open, 1795 | .release = generic_wr_fault_points_close, 1796 | .write = del_fault_points_write, 1797 | .llseek = no_llseek, 1798 | }; 1799 | 1800 | struct fault_seq { 1801 | struct fault_inject *inj; 1802 | struct fault_group *group; 1803 | struct fault_iter it; 1804 | }; 1805 | 1806 | static struct fault_point *seq_fault_iter_curr(struct fault_seq *fseq, 1807 | loff_t *pos) 1808 | { 1809 | struct fault_point *fault; 1810 | 1811 | fault = fault_iter_curr(fseq->inj, &fseq->it); 1812 | *pos = fault_iter_pos(&fseq->it); 1813 | 1814 | return fault; 1815 | } 1816 | 1817 | static struct fault_point *seq_fault_iter_next(struct fault_seq *fseq, 1818 | loff_t *pos) 1819 | { 1820 | struct fault_point *fault; 1821 | 1822 | fault = fault_iter_next(fseq->inj, &fseq->it); 1823 | *pos = fault_iter_pos(&fseq->it); 1824 | 1825 | return fault; 1826 | } 1827 | 1828 | static void *list_fault_points_seq_start(struct seq_file *sf, loff_t *pos) 1829 | { 1830 | struct fault_seq *fseq = sf->private; 1831 | bool init; 1832 | 1833 | init = fault_iter_init(fseq->inj, &fseq->it, fseq->group, *pos); 1834 | if (unlikely(!init)) 1835 | return NULL; 1836 | 1837 | if (*pos == 0) 1838 | seq_format_header(sf, fseq->inj); 1839 | 1840 | return seq_fault_iter_curr(fseq, pos); 1841 | } 1842 | 1843 | static int list_fault_points_seq_show(struct seq_file *sf, void *vp) 1844 | { 1845 | struct fault_seq *fseq = sf->private; 1846 | struct fault_point *fault = vp; 1847 | 1848 | seq_format_fault_point(sf, fseq->inj, fault, fseq->it.iter); 1849 | 1850 | return 0; 1851 | } 1852 | 1853 | static void *list_fault_points_seq_next(struct seq_file *sf, void *vp, 1854 | loff_t *pos) 1855 | { 1856 | struct fault_seq *fseq = sf->private; 1857 | 1858 | return seq_fault_iter_next(fseq, pos); 1859 | } 1860 | 1861 | static void list_fault_points_seq_stop(struct seq_file *sf, void *vp) 1862 | { 1863 | } 1864 | 1865 | static const struct seq_operations list_fault_points_seq_ops = { 1866 | .start = list_fault_points_seq_start, 1867 | .show = list_fault_points_seq_show, 1868 | .next = list_fault_points_seq_next, 1869 | .stop = list_fault_points_seq_stop, 1870 | }; 1871 | 1872 | static inline void fault_seq_init(struct fault_inject *inj, 1873 | struct fault_group *group, 1874 | struct fault_seq *fseq) 1875 | { 1876 | fseq->inj = inj; 1877 | fseq->group = group; 1878 | } 1879 | 1880 | static struct fault_seq *fault_seq_create(struct fault_inject *inj, 1881 | struct fault_group *group) 1882 | { 1883 | struct fault_seq *fseq; 1884 | 1885 | fseq = kmalloc(sizeof(*fseq), GFP_KERNEL); 1886 | if (unlikely(fseq == NULL)) 1887 | return NULL; 1888 | 1889 | fault_seq_init(inj, group, fseq); 1890 | return fseq; 1891 | } 1892 | 1893 | static int __list_fault_points_open(struct inode *inode, 1894 | struct file *file, 1895 | struct fault_inject *inj, 1896 | struct fault_group *group) 1897 | { 1898 | struct seq_file *sf; 1899 | int rc; 1900 | 1901 | rc = seq_open(file, &list_fault_points_seq_ops); 1902 | if (unlikely(rc)) 1903 | return rc; 1904 | 1905 | sf = file->private_data; 1906 | sf->private = fault_seq_create(inj, group); 1907 | if (unlikely(sf->private == NULL)) { 1908 | seq_release(inode, file); 1909 | return -ENOMEM; 1910 | } 1911 | 1912 | return nonseekable_open(inode, file); 1913 | } 1914 | 1915 | static int list_fault_points_root_open(struct inode *inode, struct file *file) 1916 | { 1917 | struct dentry *dentry = file->f_path.dentry; 1918 | struct fault_inject *inj; 1919 | int rc; 1920 | 1921 | inj = fault_fs_ref_get(dentry, ref_from_fault_inject); 1922 | if (unlikely(!inj)) 1923 | return -ENOENT; 1924 | 1925 | rc = __list_fault_points_open(inode, file, inj, NULL); 1926 | if (unlikely(rc)) 1927 | fault_fs_ref_put(&inj->fs_ref); 1928 | 1929 | return rc; 1930 | } 1931 | 1932 | static int list_fault_points_group_open(struct inode *inode, struct file *file) 1933 | { 1934 | struct dentry *dentry = file->f_path.dentry; 1935 | struct fault_group *group; 1936 | int rc; 1937 | 1938 | group = fault_fs_ref_get(dentry, ref_from_fault_group); 1939 | if (unlikely(!group)) 1940 | return -ENOENT; 1941 | 1942 | rc = __list_fault_points_open(inode, file, group->inj, group); 1943 | if (unlikely(rc)) 1944 | fault_fs_ref_put(&group->fs_ref); 1945 | 1946 | return rc; 1947 | } 1948 | 1949 | static int list_fault_points_close(struct inode *inode, struct file *file) 1950 | { 1951 | struct seq_file *sf = file->private_data; 1952 | struct fault_seq *fseq = sf->private; 1953 | 1954 | seq_release(inode, file); 1955 | if (fseq->group) 1956 | fault_fs_ref_put(&fseq->group->fs_ref); 1957 | else 1958 | fault_fs_ref_put(&fseq->inj->fs_ref); 1959 | kfree(fseq); 1960 | 1961 | return 0; 1962 | } 1963 | 1964 | static const struct file_operations list_fault_points_root_fops = { 1965 | .owner = THIS_MODULE, 1966 | .open = list_fault_points_root_open, 1967 | .release = list_fault_points_close, 1968 | .read = seq_read, 1969 | .llseek = seq_lseek, 1970 | }; 1971 | 1972 | static const struct file_operations list_fault_points_group_fops = { 1973 | .owner = THIS_MODULE, 1974 | .open = list_fault_points_group_open, 1975 | .release = list_fault_points_close, 1976 | .read = seq_read, 1977 | .llseek = seq_lseek, 1978 | }; 1979 | 1980 | static int next_group_read_op(void *data, u64 *val) 1981 | { 1982 | struct fault_inject *inj = data; 1983 | unsigned int bit; 1984 | 1985 | bit = find_first_zero_bit(inj->groups, FAULT_GROUPS_SZ); 1986 | if (bit >= FAULT_GROUPS_SZ) 1987 | return -EIO; 1988 | *val = bit; 1989 | 1990 | return 0; 1991 | } 1992 | FAULT_SIMPLE_ATTRIBUTE(next_group_fops, next_group_read_op, 1993 | NULL, "%llu\n", 1994 | ref_from_fault_inject); 1995 | 1996 | static int group_tie_with_debugfs(struct fault_group *group); 1997 | static void __group_wipe(struct fault_group *group); 1998 | 1999 | static int create_group_write_op(void *data, u64 val) 2000 | { 2001 | struct fault_inject *inj = data; 2002 | struct fault_group *group; 2003 | unsigned int bit; 2004 | int rc; 2005 | 2006 | if (val >= FAULT_GROUPS_SZ) 2007 | return -EINVAL; 2008 | 2009 | bit = val; 2010 | if (test_and_set_bit(bit, inj->groups)) 2011 | return -EINVAL; 2012 | 2013 | group = group_allocate(inj, bit); 2014 | if (unlikely(group == NULL)) { 2015 | clear_bit(bit, inj->groups); 2016 | return -EIO; 2017 | } 2018 | 2019 | 2020 | /* 2021 | * We have to get the reference to invoke normal group_put. 2022 | * See comments below. 2023 | */ 2024 | __group_get(group); 2025 | rc = group_tie_with_debugfs(group); 2026 | if (unlikely(rc)) { 2027 | /* 2028 | * The thing is that debugfs entries can be partially created 2029 | * and someone can already put faults to that group and code 2030 | * can start to execute faults. All we need is to wipe the 2031 | * group in generic way, i.e. traverse the faults and unlink 2032 | * them, and then put the group reference. For sure reference 2033 | * can be not zero and actual freeing will happen when fault 2034 | * code puts the last reference. 2035 | */ 2036 | __group_wipe(group); 2037 | group_put(group); 2038 | return -EIO; 2039 | } 2040 | group_list_add_and_get(group); 2041 | group_put(group); 2042 | 2043 | return 0; 2044 | } 2045 | FAULT_SIMPLE_ATTRIBUTE(create_group_fops, NULL, 2046 | create_group_write_op, "%llu\n", 2047 | ref_from_fault_inject); 2048 | 2049 | static int group_wipe(struct fault_group *group); 2050 | 2051 | static int delete_group_write_op(void *data, u64 val) 2052 | { 2053 | struct fault_inject *inj = data; 2054 | struct fault_group *group; 2055 | unsigned int bit; 2056 | int rc; 2057 | 2058 | if (val >= FAULT_GROUPS_SZ) 2059 | return -EINVAL; 2060 | 2061 | bit = val; 2062 | group = group_find_and_get(inj, bit); 2063 | if (unlikely(!group)) 2064 | return -EINVAL; 2065 | rc = group_wipe(group); 2066 | group_put(group); 2067 | 2068 | return rc; 2069 | } 2070 | FAULT_SIMPLE_ATTRIBUTE(delete_group_fops, NULL, 2071 | delete_group_write_op, "%llu\n", 2072 | ref_from_fault_inject); 2073 | 2074 | struct fault_debugfs_entry { 2075 | const char *name; 2076 | umode_t mode; 2077 | const struct file_operations *fops; 2078 | }; 2079 | 2080 | #ifdef CONFIG_FAULT_INJECTION 2081 | #define TASK_FILTER_ENTRY \ 2082 | /* This is Linux kernel configuration, see /proc/{pid}/make-it-fail */ \ 2083 | {"task_filter", S_IWUSR | S_IRUGO, &task_filter_fault_fops}, 2084 | #else 2085 | #define TASK_FILTER_ENTRY 2086 | #endif 2087 | 2088 | #define COMMON_FAULT_ENTRIES \ 2089 | TASK_FILTER_ENTRY \ 2090 | {"enable", S_IWUSR | S_IRUGO, &enable_fault_fops }, \ 2091 | {"injected", S_IRUGO, &injected_fault_fops }, \ 2092 | {"hits", S_IRUGO, &hits_fault_fops }, \ 2093 | {"times", S_IWUSR | S_IRUGO, ×_fault_fops }, \ 2094 | {"probability", S_IWUSR | S_IRUGO, &probability_fault_fops }, \ 2095 | {"interval", S_IWUSR | S_IRUGO, &interval_fault_fops } 2096 | 2097 | #define NULL_ENTRY {NULL, 0, NULL} 2098 | 2099 | struct fault_debugfs_entry root_entries[] = { 2100 | {"list_fault_points", S_IRUGO, &list_fault_points_root_fops }, 2101 | {"next_group", S_IRUGO, &next_group_fops }, 2102 | {"create_group", S_IWUSR, &create_group_fops }, 2103 | {"delete_group", S_IWUSR, &delete_group_fops }, 2104 | NULL_ENTRY 2105 | }; 2106 | 2107 | struct fault_debugfs_entry group_entries[] = { 2108 | {"list_fault_points", S_IRUGO, &list_fault_points_group_fops }, 2109 | {"add_fault_points", S_IWUSR, &add_fault_points_fops }, 2110 | {"del_fault_points", S_IWUSR, &del_fault_points_fops }, 2111 | NULL_ENTRY 2112 | }; 2113 | 2114 | struct fault_debugfs_entry delay_fault_entries[] = { 2115 | COMMON_FAULT_ENTRIES, 2116 | {"delay_us", S_IWUSR | S_IRUGO, &delay_us_fault_fops }, 2117 | NULL_ENTRY 2118 | }; 2119 | 2120 | struct fault_debugfs_entry error_fault_entries[] = { 2121 | COMMON_FAULT_ENTRIES, 2122 | {"errors", S_IWUSR | S_IRUGO, &errors_fault_fops }, 2123 | NULL_ENTRY 2124 | }; 2125 | 2126 | struct fault_debugfs_entry panic_fault_entries[] = { 2127 | COMMON_FAULT_ENTRIES, 2128 | NULL_ENTRY 2129 | }; 2130 | 2131 | /* Keep the order in sync with fault_type and friends */ 2132 | struct fault_debugfs_entry *all_fault_entries[] = { 2133 | delay_fault_entries, 2134 | error_fault_entries, 2135 | panic_fault_entries 2136 | }; 2137 | 2138 | static struct dentry *fault_create_debugfs_entries( 2139 | struct fault_fs_ref *fs_ref, 2140 | struct fault_debugfs_entry *entries, 2141 | struct dentry **dentries_out, 2142 | const char *name, struct dentry *parent, 2143 | void *priv) 2144 | { 2145 | int i; 2146 | struct dentry *root = NULL, *d; 2147 | struct fault_debugfs_entry *e; 2148 | 2149 | d = debugfs_create_dir(name, parent); 2150 | if (unlikely(IS_ERR_OR_NULL(d))) 2151 | goto error; 2152 | 2153 | root = d; 2154 | 2155 | for (i = 0; ; i++) { 2156 | if (entries[i].name == NULL) 2157 | break; 2158 | 2159 | e = &entries[i]; 2160 | d = debugfs_create_file(e->name, e->mode, root, priv, e->fops); 2161 | if (unlikely(IS_ERR_OR_NULL(d))) 2162 | goto error; 2163 | if (dentries_out) 2164 | dentries_out[i] = d; 2165 | } 2166 | 2167 | return root; 2168 | 2169 | error: 2170 | if (!IS_ERR(d)) 2171 | d = ERR_PTR(-ENOMEM); 2172 | 2173 | fault_dentry_remove(fs_ref, root); 2174 | return d; 2175 | } 2176 | 2177 | static int group_tie_with_debugfs(struct fault_group *group) 2178 | { 2179 | int i = 0; 2180 | struct fault_cfg *cfg; 2181 | struct fault_debugfs_entry *fault_entries; 2182 | struct fault_inject *inj = group->inj; 2183 | struct dentry *dentry; 2184 | char name[32]; 2185 | 2186 | snprintf(name, sizeof(name), "%u", group->id); 2187 | dentry = fault_create_debugfs_entries(&group->fs_ref, group_entries, 2188 | NULL, name, inj->root_dentry, 2189 | group); 2190 | if (unlikely(IS_ERR(dentry))) 2191 | return PTR_ERR(dentry); 2192 | 2193 | group->dentry = dentry; 2194 | 2195 | BUILD_BUG_ON(ARRAY_SIZE(group->faults) != ARRAY_SIZE(all_fault_entries)); 2196 | 2197 | for (i = 0; i < ARRAY_SIZE(group->faults); i++) { 2198 | cfg = &group->faults[i]; 2199 | fault_entries = all_fault_entries[i]; 2200 | 2201 | dentry = fault_create_debugfs_entries(&group->fs_ref, 2202 | fault_entries, NULL, 2203 | cfg->name, group->dentry, 2204 | cfg); 2205 | if (unlikely(IS_ERR(dentry))) { 2206 | fault_dentry_remove(&group->fs_ref, group->dentry); 2207 | group->dentry = NULL; 2208 | return PTR_ERR(dentry); 2209 | } 2210 | 2211 | cfg->dentry = dentry; 2212 | } 2213 | 2214 | return 0; 2215 | } 2216 | 2217 | /** 2218 | * group_wipe() - prepares group for complete deletion 2219 | * 2220 | * Deletes group from list, unties group from debugfs, unlinks 2221 | * group from each fault point, clears group bit from bitmap. 2222 | * 2223 | * After this sequence group is ready for last reference put. 2224 | * For sure caller must hold the reference on this group, so 2225 | * the caller is responsible for final put. 2226 | */ 2227 | static void __group_wipe(struct fault_group *group) 2228 | { 2229 | int rc; 2230 | 2231 | BUG_ON(!list_empty(&group->list)); 2232 | 2233 | rc = for_each_fault_inject_entry( 2234 | group->inj, group_unlink_from_fault, group); 2235 | BUG_ON(rc); 2236 | 2237 | rc = test_and_clear_bit(group->id, group->inj->groups); 2238 | BUG_ON(!rc); 2239 | } 2240 | 2241 | static int group_wipe(struct fault_group *group) 2242 | { 2243 | if (!group_list_del_and_put(group)) 2244 | return -EINVAL; 2245 | 2246 | /* 2247 | * We are alone here, group was successfully removed from the list, 2248 | * so nobody can access and remove it twice. 2249 | */ 2250 | 2251 | fault_dentry_remove(&group->fs_ref, group->dentry); 2252 | __group_wipe(group); 2253 | 2254 | return 0; 2255 | } 2256 | 2257 | /** 2258 | * group_delete_all() - completely deletes all groups. 2259 | * 2260 | * Iterates over group list and wipes each group one by one, 2261 | * then waits for last reference and makes final put. The caller 2262 | * should guarantee that nobody will come and create/delete/modify 2263 | * groups while we are here. 2264 | */ 2265 | static void group_delete_all(struct fault_inject *inj) 2266 | { 2267 | int rc; 2268 | bool done; 2269 | unsigned int bit; 2270 | struct fault_group *group, *t; 2271 | 2272 | spin_lock_irq(&inj->lock); 2273 | list_for_each_entry_safe(group, t, &inj->list_groups, list) { 2274 | __group_get(group); 2275 | BUG_ON(group->inj != inj); 2276 | spin_unlock_irq(&inj->lock); 2277 | 2278 | rc = group_wipe(group); 2279 | BUG_ON(rc); 2280 | group_wait_last_ref(group); 2281 | done = group_put(group); 2282 | BUG_ON(!done); 2283 | 2284 | spin_lock_irq(&inj->lock); 2285 | } 2286 | spin_unlock_irq(&inj->lock); 2287 | 2288 | /* Check all bits are cleared */ 2289 | bit = find_first_bit(inj->groups, FAULT_GROUPS_SZ); 2290 | BUG_ON(bit < FAULT_GROUPS_SZ); 2291 | BUG_ON(!list_empty(&inj->list_groups)); 2292 | } 2293 | 2294 | static void fault_inject_cleanup(struct fault_inject *inj) 2295 | { 2296 | int i; 2297 | 2298 | /* 2299 | * Delete root dentries first to be sure nobody will 2300 | * come and create/delete groups 2301 | */ 2302 | for (i = 0; i < inj->attr_dent_num; i++) 2303 | fault_dentry_remove(&inj->fs_ref, inj->attr_dentries[i]); 2304 | kfree(inj->attr_dentries); 2305 | group_delete_all(inj); 2306 | fault_dentry_remove(&inj->fs_ref, inj->root_dentry); 2307 | } 2308 | 2309 | static struct dentry *fi_dentry; 2310 | 2311 | int fault_inject_register(struct fault_inject *inj, struct module *mod) 2312 | { 2313 | int rc; 2314 | 2315 | inj->inited = false; 2316 | if (unlikely(IS_ERR_OR_NULL(fi_dentry))) 2317 | return -ENOTSUPP; 2318 | 2319 | inj->mod = mod; 2320 | inj->max_func_sz = sizeof(FUNC_HEADER) - 1; 2321 | spin_lock_init(&inj->lock); 2322 | INIT_LIST_HEAD(&inj->list_groups); 2323 | fault_fs_ref_init(&inj->fs_ref); 2324 | bitmap_zero(inj->groups, FAULT_GROUPS_SZ); 2325 | 2326 | inj->attr_dent_num = ARRAY_SIZE(root_entries) - 1; 2327 | inj->attr_dentries = kmalloc_array(inj->attr_dent_num, 2328 | sizeof(*inj->attr_dentries), 2329 | GFP_KERNEL); 2330 | if (unlikely(inj->attr_dentries == NULL)) 2331 | return -ENOMEM; 2332 | 2333 | inj->root_dentry = 2334 | fault_create_debugfs_entries(&inj->fs_ref, root_entries, 2335 | inj->attr_dentries, 2336 | mod->name, fi_dentry, inj); 2337 | if (unlikely(IS_ERR(inj->root_dentry))) { 2338 | kfree(inj->attr_dentries); 2339 | return PTR_ERR(inj->root_dentry); 2340 | } 2341 | 2342 | /* 2343 | * Dry run each fault entry to setup correct max size of the 2344 | * function to have pretty format 2345 | */ 2346 | rc = for_each_fault_inject_entry( 2347 | inj, dry_format_fault_point, NULL); 2348 | if (unlikely(rc)) 2349 | goto err; 2350 | 2351 | /* Finally done */ 2352 | inj->inited = true; 2353 | return 0; 2354 | 2355 | err: 2356 | fault_inject_cleanup(inj); 2357 | return rc; 2358 | } 2359 | EXPORT_SYMBOL_GPL(fault_inject_register); 2360 | 2361 | void fault_inject_unregister(struct fault_inject *inj) 2362 | { 2363 | if (likely(!IS_ERR_OR_NULL(inj) && inj->inited)) { 2364 | inj->inited = false; 2365 | fault_inject_cleanup(inj); 2366 | } 2367 | } 2368 | EXPORT_SYMBOL_GPL(fault_inject_unregister); 2369 | 2370 | static int fault_inject_init(void) 2371 | { 2372 | fi_dentry = debugfs_create_dir("fault_inject", NULL); 2373 | if (unlikely(IS_ERR_OR_NULL(fi_dentry))) 2374 | return -ENOTSUPP; 2375 | return 0; 2376 | } 2377 | 2378 | static void fault_inject_exit(void) 2379 | { 2380 | BUG_ON(fi_dentry == NULL); 2381 | debugfs_remove(fi_dentry); 2382 | } 2383 | 2384 | module_init(fault_inject_init); 2385 | module_exit(fault_inject_exit); 2386 | 2387 | MODULE_AUTHOR("Roman Pen "); 2388 | MODULE_DESCRIPTION("Improved fault injection framework"); 2389 | MODULE_LICENSE("GPL"); 2390 | MODULE_VERSION(__stringify(BUILD_VERSION)); 2391 | -------------------------------------------------------------------------------- /fault-injection.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Fault injection framework. 3 | * 4 | * Copyright (c) 2018 ProfitBricks GmbH. 5 | * Authors: Roman Pen 6 | * 7 | * This program is free software; you can redistribute it and/or 8 | * modify it under the terms of the GNU General Public License as 9 | * published by the Free Software Foundation, version 2 of the 10 | * License. 11 | */ 12 | 13 | #ifndef __FAULT_INJECT_H 14 | #define __FAULT_INJECT_H 15 | 16 | #include 17 | #include 18 | #include 19 | 20 | #define FAULT_ERRORS_SZ 256 21 | #define FAULT_GROUPS_SZ 256 22 | #define FAULT_MAGIC 0xa4dbd0dd 23 | 24 | struct fault_inject; 25 | 26 | typedef int (inj_fn)(struct fault_inject *inj, void *addr); 27 | typedef int (reg_fn)(struct fault_inject *inj, struct module *mod); 28 | typedef int (unreg_fn)(struct fault_inject *inj); 29 | 30 | struct fault_funcs { 31 | inj_fn *inj; 32 | reg_fn *reg; 33 | unreg_fn *unreg; 34 | }; 35 | 36 | struct fault_fs_ref { 37 | wait_queue_head_t wait; 38 | atomic_t ref; 39 | }; 40 | 41 | struct fault_inject { 42 | struct fault_fs_ref fs_ref; 43 | struct module *mod; 44 | struct dentry *root_dentry; 45 | struct dentry **attr_dentries; 46 | unsigned int attr_dent_num; 47 | struct list_head list_groups; 48 | DECLARE_BITMAP( groups, FAULT_GROUPS_SZ); 49 | spinlock_t lock; 50 | unsigned int max_func_sz; 51 | bool inited; 52 | struct fault_funcs fn; 53 | }; 54 | 55 | #ifndef FAULT_INJECT_MODULE 56 | 57 | static inline int fault_inject_register(struct fault_inject *inj, 58 | struct module *mod) 59 | { 60 | inj->fn.inj = __symbol_get("inject_faults_by_target"); 61 | if (unlikely(inj->fn.inj == NULL)) 62 | goto err_inj_get; 63 | inj->fn.reg = __symbol_get("fault_inject_register"); 64 | if (unlikely(inj->fn.reg == NULL)) 65 | goto err_reg_get; 66 | inj->fn.unreg = __symbol_get("fault_inject_unregister"); 67 | if (unlikely(inj->fn.unreg == NULL)) 68 | goto err_unreg_get; 69 | return inj->fn.reg(inj, mod); 70 | 71 | err_unreg_get: 72 | __symbol_put("fault_inject_register"); 73 | err_reg_get: 74 | __symbol_put("inject_faults_by_target"); 75 | err_inj_get: 76 | inj->inited = false; 77 | return -ENODEV; 78 | } 79 | 80 | static inline void fault_inject_unregister(struct fault_inject *inj) 81 | { 82 | if (likely(!IS_ERR_OR_NULL(inj) && inj->inited)) { 83 | inj->fn.unreg(inj); 84 | BUG_ON(inj->inited); 85 | __symbol_put("fault_inject_unregister"); 86 | __symbol_put("fault_inject_register"); 87 | __symbol_put("inject_faults_by_target"); 88 | } 89 | } 90 | 91 | static inline int inject_faults_by_target(struct fault_inject *inj, 92 | void *addr) 93 | { 94 | BUG_ON(inj->fn.inj == NULL); 95 | return inj->fn.inj(inj, addr); 96 | } 97 | 98 | /** 99 | * The following repeats jump_label.h macro with only one major difference: 100 | * we do not depend on static_key in .bss, we create key on demand. The 101 | * problem is that if caller function is inlined and is invoked from many 102 | * places static key in its turn is declared only once and thus we get 103 | * '1 key = N fault points' dependency. To have exact match 1 key = 1 fault 104 | * we do this magic. 105 | */ 106 | static __always_inline void * 107 | arch_fault_static_branch(const char *class, const char *file, unsigned int line) 108 | { 109 | void *addr = NULL; 110 | const unsigned key_sz = sizeof(struct static_key); 111 | 112 | asm_volatile_goto( 113 | "1:" 114 | ".byte " __stringify(STATIC_KEY_INIT_NOP) "\n\t" 115 | ".pushsection .fault_inject, \"aw\" \n\t" 116 | _ASM_ALIGN "\n\t" 117 | "666:\n\t" 118 | ".fill %c0, 1, 0x00\n\t" /* zero key */ 119 | ".long " __stringify(FAULT_MAGIC) "\n\t" /* magic */ 120 | ".long %c1 \n\t" /* line number */ 121 | _ASM_PTR "%c2 \n\t" /* file ptr */ 122 | _ASM_PTR "%c3 \n\t" /* class ptr */ 123 | _ASM_PTR "0x00 \n\t" /* group ptr */ 124 | ".popsection\n\t" 125 | ".pushsection __jump_table, \"aw\" \n\t" 126 | _ASM_ALIGN "\n\t" 127 | _ASM_PTR "1b, %l[l_yes], 666b \n\t" 128 | ".popsection \n\t" 129 | : 130 | : "i"(key_sz), "i"(line), "i"(file), "i"(class) 131 | : 132 | : l_yes); 133 | 134 | return NULL; 135 | l_yes: 136 | /* 137 | * Unfortunately simple expression 'addr = &&l_yes;' does not 138 | * work and instead of getting pointer exactly on l_yes label: 139 | * "lea -0x7(%rip), %rsi", i.e. look back on 7 bytes of current 140 | * %rip, gcc generates "lea 0x0(%rip), %rsi". Ok, gcc, we expect 141 | * assembler still works. 142 | */ 143 | asm volatile("1: lea 1b(%%rip),%0\n\t" 144 | : "=r" (addr)); 145 | 146 | return addr; 147 | } 148 | 149 | /** 150 | * INJECT_FAULTS() - Fault injection macro. 151 | * 152 | * Injects fault. If error was injected, the error code will be returned. 153 | * If delay was injected, zero should be returned. 154 | */ 155 | #define INJECT_FAULT(inj, class) ({ \ 156 | int err = 0; \ 157 | void *true_br_addr; \ 158 | \ 159 | true_br_addr = arch_fault_static_branch(class, __FILE__, __LINE__); \ 160 | if (unlikely(true_br_addr)) \ 161 | /* slow path */ \ 162 | err = inject_faults_by_target(inj, true_br_addr); \ 163 | err; \ 164 | }) 165 | 166 | /** 167 | * INJECT_FAULT_INT() - Fault injection macro. 168 | * 169 | * Injects fault. If fault was successfully injected, the 'func' parameter 170 | * is not called, and errno is returned. If fault was not fired - function 171 | * is called and its returning value will be returned as an integer. 172 | */ 173 | #define INJECT_FAULT_INT(inj, class, func) ({ \ 174 | int err; \ 175 | \ 176 | err = INJECT_FAULT(inj, class); \ 177 | if (likely(!err)) \ 178 | err = func; \ 179 | err; \ 180 | }) 181 | 182 | /** 183 | * INJECT_FAULT_NULL_PTR() - Fault injection macro. 184 | * 185 | * Injects fault. If fault was successfully injected, the 'func' parameter 186 | * is not called, and NULL pointer is returned. If fault was not fired - 187 | * function is called and its returning value will be returned as a pointer. 188 | */ 189 | #define INJECT_FAULT_NULL_PTR(inj, class, func) ({ \ 190 | int err; \ 191 | void *p; \ 192 | \ 193 | err = INJECT_FAULT(inj, class); \ 194 | if (unlikely(err)) \ 195 | p = NULL; \ 196 | else \ 197 | p = func; \ 198 | p; \ 199 | }) 200 | 201 | /** 202 | * INJECT_FAULT_ERR_PTR() - Fault injection macro. 203 | * 204 | * Injects fault. If fault was successfully injected, the 'func' parameter 205 | * is not called, and errno inside pointer is returned. If fault was not 206 | * fired - function is called and its returning value will be returned as 207 | * a pointer. 208 | */ 209 | #define INJECT_FAULT_ERR_PTR(inj, class, func) ({ \ 210 | int err; \ 211 | void *p; \ 212 | \ 213 | err = INJECT_FAULT(inj, class); \ 214 | if (unlikely(err)) \ 215 | p = ERR_PTR(err); \ 216 | else \ 217 | p = func; \ 218 | p; \ 219 | }) 220 | 221 | #endif /* FAULT_INJECT_MODULE */ 222 | #endif /* !__FAULT_INJECT_H */ 223 | -------------------------------------------------------------------------------- /tests/.gitignore: -------------------------------------------------------------------------------- 1 | test_fault_inject.h 2 | Module.symvers 3 | modules.order 4 | .tmp_versions 5 | *~ 6 | *.ko 7 | *.cmd 8 | *.mod.c 9 | *.o 10 | -------------------------------------------------------------------------------- /tests/Makefile: -------------------------------------------------------------------------------- 1 | # 2 | # Copyright Profitbricks GmbH - All rights reserved 3 | # 4 | 5 | # Called as kbuild 6 | ifneq ($(KERNELRELEASE),) 7 | 8 | # We are not a part of the kernel, thus build always as a module 9 | obj-m := test_fault_inject.o 10 | 11 | KBUILD_EXTRA_SYMBOLS := $(src)/../Module.symvers 12 | 13 | # Normal Makefile, redirect to kbuild 14 | else 15 | 16 | KDIR ?= /usr/src/linux 17 | ROOT_DIR := $(PWD) 18 | 19 | default:: 20 | @$(MAKE) -C $(KDIR) M=$(ROOT_DIR) $(MAKEFLAGS) 21 | 22 | clean:: 23 | @$(MAKE) -C $(KDIR) M=$(ROOT_DIR) $(MAKEFLAGS) clean 24 | 25 | install:: default 26 | 27 | # By default modules_install does not build anything. That's odd. 28 | # So not to break previous behavior I simply call modules build before 29 | # installing them. 30 | modules_install:: default 31 | @$(MAKE) -C $(KDIR) M=$(ROOT_DIR) $(MAKEFLAGS) modules_install 32 | 33 | endif 34 | -------------------------------------------------------------------------------- /tests/test_fault_inject.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Test scenarious for fault injection framework. 3 | * 4 | * Copyright (C) 2015 Roman Pen 5 | * 6 | * This program is free software; you can redistribute it and/or 7 | * modify it under the terms of the GNU General Public License as 8 | * published by the Free Software Foundation, version 2 of the 9 | * License. 10 | * 11 | * If you have changed something, please do not forget to reflec 12 | * these changes in fault injection documentation file: README 13 | */ 14 | 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | 23 | #include 24 | #include 25 | 26 | #include "../fault_inject.h" 27 | 28 | static struct dentry *test_dentry; 29 | static struct fault_inject inj; 30 | static struct task_struct *thr; 31 | static bool reged; 32 | static unsigned long long injected; 33 | static DEFINE_MUTEX(lock); 34 | 35 | #include "test_fault_inject.h" 36 | 37 | static int register_write_op(void *data, u64 val) 38 | { 39 | int rc = -EINVAL; 40 | 41 | mutex_lock(&lock); 42 | if (unlikely(reged)) 43 | goto out; 44 | rc = fault_inject_register(&inj, THIS_MODULE); 45 | reged = !rc; 46 | out: 47 | mutex_unlock(&lock); 48 | 49 | return rc; 50 | } 51 | DEFINE_SIMPLE_ATTRIBUTE(register_fops, NULL, register_write_op, "%llu\n"); 52 | 53 | static int unregister_write_op(void *data, u64 val) 54 | { 55 | int rc = -EINVAL; 56 | 57 | mutex_lock(&lock); 58 | if (unlikely(!reged)) 59 | goto out; 60 | fault_inject_unregister(&inj); 61 | reged = false; 62 | rc = 0; 63 | out: 64 | mutex_unlock(&lock); 65 | 66 | return rc; 67 | } 68 | DEFINE_SIMPLE_ATTRIBUTE(unregister_fops, NULL, unregister_write_op, "%llu\n"); 69 | 70 | static __maybe_unused int fault_function(void) 71 | { 72 | int rc = 0; 73 | 74 | mutex_lock(&lock); 75 | if (reged) 76 | rc = INJECT_FAULT(&inj, NULL); 77 | mutex_unlock(&lock); 78 | 79 | return rc; 80 | } 81 | 82 | static int run_fault(void *notused) 83 | { 84 | int rc; 85 | 86 | while (!kthread_should_stop()) { 87 | CALL_FAULTS(); 88 | } 89 | return 0; 90 | } 91 | 92 | static int start_thread_write_op(void *data, u64 val) 93 | { 94 | int rc = -EINVAL; 95 | 96 | mutex_lock(&lock); 97 | 98 | if (unlikely(thr)) 99 | goto out; 100 | thr = kthread_run(run_fault, NULL, "test_fault_injection"); 101 | BUG_ON(IS_ERR(thr)); 102 | rc = 0; 103 | out: 104 | mutex_unlock(&lock); 105 | 106 | return rc; 107 | } 108 | DEFINE_SIMPLE_ATTRIBUTE(start_thread_fops, NULL, start_thread_write_op, "%llu\n"); 109 | 110 | static int stop_thread_write_op(void *data, u64 val) 111 | { 112 | struct task_struct *t_thr; 113 | 114 | mutex_lock(&lock); 115 | t_thr = thr; 116 | thr = NULL; 117 | mutex_unlock(&lock); 118 | 119 | if (unlikely(!t_thr)) 120 | return -EINVAL; 121 | 122 | kthread_stop(t_thr); 123 | 124 | return 0; 125 | } 126 | DEFINE_SIMPLE_ATTRIBUTE(stop_thread_fops, NULL, stop_thread_write_op, "%llu\n"); 127 | 128 | static int faults_injected_read_op(void *data, u64 *val) 129 | { 130 | *val = injected; 131 | 132 | return 0; 133 | } 134 | DEFINE_SIMPLE_ATTRIBUTE(faults_injected_fops, faults_injected_read_op, NULL, "%llu\n"); 135 | 136 | static int test_fault_inject_init(void) 137 | { 138 | struct dentry *d; 139 | 140 | test_dentry = debugfs_create_dir("test_fault_inject", NULL); 141 | if (unlikely(IS_ERR_OR_NULL(test_dentry))) 142 | return -ENOTSUPP; 143 | 144 | d = debugfs_create_file("register", S_IWUSR, test_dentry, 145 | NULL, ®ister_fops); 146 | BUG_ON(IS_ERR_OR_NULL(d)); 147 | 148 | d = debugfs_create_file("unregister", S_IWUSR, test_dentry, 149 | NULL, &unregister_fops); 150 | BUG_ON(IS_ERR_OR_NULL(d)); 151 | 152 | d = debugfs_create_file("start_thread", S_IWUSR, test_dentry, 153 | NULL, &start_thread_fops); 154 | BUG_ON(IS_ERR_OR_NULL(d)); 155 | 156 | d = debugfs_create_file("stop_thread", S_IWUSR, test_dentry, 157 | NULL, &stop_thread_fops); 158 | BUG_ON(IS_ERR_OR_NULL(d)); 159 | 160 | d = debugfs_create_file("faults_injected", S_IRUGO, test_dentry, 161 | NULL, &faults_injected_fops); 162 | BUG_ON(IS_ERR_OR_NULL(d)); 163 | 164 | return 0; 165 | } 166 | module_init(test_fault_inject_init); 167 | 168 | static void test_fault_inject_exit(void) 169 | { 170 | BUG_ON(test_dentry == NULL); 171 | debugfs_remove_recursive(test_dentry); 172 | 173 | if (thr) 174 | kthread_stop(thr); 175 | if (reged) 176 | fault_inject_unregister(&inj); 177 | 178 | } 179 | module_exit(test_fault_inject_exit); 180 | 181 | MODULE_AUTHOR("Roman Pen "); 182 | MODULE_DESCRIPTION("Some test scenarious for improved fault injection framework"); 183 | MODULE_LICENSE("GPL"); 184 | -------------------------------------------------------------------------------- /tests/test_fault_inject.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | rand_range() { 4 | x=$1 5 | y=$2 6 | echo $(($x + $RANDOM % ($y-$x+1))) 7 | } 8 | 9 | dd_files() { 10 | src=$1 11 | dst=$2 12 | 13 | src_bs=$(rand_range 1 4097) 14 | dst_bs=$(rand_range 1 4097) 15 | 16 | (dd if=$src bs=$src_bs | dd of=$dst bs=$dst_bs) >/dev/null 2>&1 17 | } 18 | 19 | DIR=$(dirname $0) 20 | ITERS=1000 21 | CALLS=1000 22 | 23 | F=/sys/kernel/debug/fault_inject/test_fault_inject 24 | T=/sys/kernel/debug/test_fault_inject 25 | 26 | FUNC=$(cat <> $DIR/test_fault_inject.h 52 | done 53 | 54 | echo "#define CALL_FAULTS() \\" >> $DIR/test_fault_inject.h 55 | for i in `seq 1 $CALLS`; do 56 | COUNT=$i 57 | eval echo \"$CALL \\\\\" >> $DIR/test_fault_inject.h 58 | done 59 | echo >> $DIR/test_fault_inject.h 60 | 61 | # Build and insmod fault_inject 62 | #(cd $DIR/..; make) 63 | insmod $DIR/../fault_inject.ko >/dev/null 2>&1 64 | 65 | # Build and insmod test_fault-inject 66 | #(cd $DIR; make) 67 | rmmod test_fault_inject >/dev/null 2>&1 68 | insmod $DIR/test_fault_inject.ko >/dev/null 2>&1 69 | 70 | echo > $T/register 71 | echo > $T/start_thread 72 | 73 | echo "- Create/delete group $ITERS times" 74 | for i in `seq 1 $ITERS`; do 75 | echo 0 > $F/create_group 76 | echo 0 > $F/delete_group 77 | done 78 | 79 | echo "- Create group/add_faults/delete group $ITERS times" 80 | for i in `seq 1 $ITERS`; do 81 | echo 0 > $F/create_group 82 | dd_files "$F/list_fault_points" "$F/0/add_fault_points" 83 | echo 0 > $F/delete_group 84 | done 85 | 86 | echo "- Create group/add_faults/faults_enable/delete group $ITERS times" 87 | for i in `seq 1 $ITERS`; do 88 | echo 0 > $F/create_group 89 | dd_files "$F/list_fault_points" "$F/0/add_fault_points" 90 | # Errors 91 | echo -ENOMEM,-ENOENT > $F/0/error/errors 92 | echo 1 > $F/0/error/probability 93 | echo 1 > $F/0/error/enable 94 | # Delays 95 | echo 0:2000 > $F/0/delay/delay_us 96 | echo 1 > $F/0/delay/probability 97 | echo 1 > $F/0/delay/enable 98 | 99 | echo 0 > $F/delete_group 100 | done 101 | 102 | echo "- Create group/add_faults/faults_enable/del_faults/delete group $ITERS times" 103 | for i in `seq 1 $ITERS`; do 104 | echo 0 > $F/create_group 105 | dd_files "$F/list_fault_points" "$F/0/add_fault_points" 106 | # Errors 107 | echo -ENOMEM,-ENOENT > $F/0/error/errors 108 | echo 1 > $F/0/error/probability 109 | echo 1 > $F/0/error/enable 110 | # Delays 111 | echo 0:2000 > $F/0/delay/delay_us 112 | echo 1 > $F/0/delay/probability 113 | echo 1 > $F/0/delay/enable 114 | 115 | dd_files "$F/list_fault_points" "$F/0/del_fault_points" 116 | echo 0 > $F/delete_group 117 | done 118 | 119 | 120 | echo > $T/stop_thread 121 | echo > $T/unregister 122 | 123 | FAULTS=`cat $T/faults_injected` 124 | echo 125 | echo "FAULTS INJECTED: $FAULTS" 126 | 127 | rmmod test_fault_inject 128 | --------------------------------------------------------------------------------