78 |
Introduction
79 |
Since R is not really a systems-programming language some facilities present in other such languages (e.g. C/C++, Python) haven’t been yet brought to R. One of such features is process management which is understood here as the capability to create, interact with and control the lifetime of child processes.
80 |
The R package subprocess aims at filling this gap by providing the few basic operations to carry out the aforementioned tasks. The spawn_subprocess() function starts a new child process and returns a handle which can be later used in process_read() and process_write() to send and receive data or in process_wait() or process_terminate() to stop the such a process.
81 |
The R subprocess package has been designed after the exemplary Python package which goes by the same. Its documentation can be found here and numerous examples of how it can be used can be found on the Web.
82 |
The R subprocess package has been verified to run on Linux, Windows and MacOS.
83 |
85 |
Design and Implementation
86 |
The main concept in the package is the handle which holds process identifiers and an external pointer object which in turn is a handle to a low-level data structure holding various system-level parameters of the running sub-process.
87 |
A child process, once started, runs until it exits on its own or until its killed. Its current state as well as its exit status can be obtained by dedicated API.
88 |
Communication with the child process can be carried out over the three standard streams: the standard input, standard output and standard error output. These streams are intercepted on the child process’ side and redirected into three anonymous pipes whose other ends are held by the parent process and can be accessed through the process handle.
89 |
In Linux these are regular pipes created with the pipe() system call and opened in the non-blocking mode. All communication takes place on request and follows the usual OS rules (e.g. the sub-process will sleep if its output buffer gets filled).
90 |
In Windows these pipes are created with the CreatePipe() call and opened in the blocking mode. Windows does not support non-blocking (overlapped in Windows-speak) mode for anonymous pipes. For that reason each stream has an accompanying reader thread. Reader threads are separated from R interpreter, do not exchange memory with the R interpreter and will not break the single-thread assumption under which R operates.
91 |
93 |
Introduction to Examples
94 |
Before we move on to examples, let’s define a few helper functions that abstract out the underlying operating system. We will use them throughout this vignette.
95 |
is_windows <- function () (tolower(.Platform$OS.type) == "windows")
96 |
97 | R_binary <- function () {
98 | R_exe <- ifelse (is_windows(), "R.exe", "R")
99 | return(file.path(R.home("bin"), R_exe))
100 | }
101 |
Just for the record, vignette has been built in Linux.
102 |
ifelse(is_windows(), "Windows", "Linux")
103 | #> [1] "Linux"
104 |
Now we can load the package and move on to the next section.
105 |
106 |
108 |
Example: controlling chlid R process
109 |
In this example we spawn a new R process, send a few commands to its standard input and read the responses from its standard output. First, let’s spawn the child process (and give it a moment to complete the start-up sequence):
110 |
handle <- spawn_process(R_binary(), c('--no-save'))
111 | Sys.sleep(1)
112 |
Let’s see the description of the child process:
113 |
print(handle)
114 | #> Process Handle
115 | #> command : /usr/lib/R/bin/R --no-save
116 | #> system id : 14227
117 | #> state : running
118 |
And now let’s see what we can find it the child’s output:
119 |
process_read(handle, PIPE_STDOUT, timeout = 1000)
120 | #> [1] ""
121 | #> [2] "R version 3.4.1 (2017-06-30) -- \"Single Candle\""
122 | #> [3] "Copyright (C) 2017 The R Foundation for Statistical Computing"
123 | #> [4] "Platform: x86_64-pc-linux-gnu (64-bit)"
124 | #> [5] ""
125 | #> [6] "R is free software and comes with ABSOLUTELY NO WARRANTY."
126 | #> [7] "You are welcome to redistribute it under certain conditions."
127 | #> [8] "Type 'license()' or 'licence()' for distribution details."
128 | #> [9] ""
129 | #> [10] " Natural language support but running in an English locale"
130 | #> [11] ""
131 | #> [12] "R is a collaborative project with many contributors."
132 | #> [13] "Type 'contributors()' for more information and"
133 | #> [14] "'citation()' on how to cite R or R packages in publications."
134 | #> [15] ""
135 | #> [16] "Type 'demo()' for some demos, 'help()' for on-line help, or"
136 | #> [17] "'help.start()' for an HTML browser interface to help."
137 | #> [18] "Type 'q()' to quit R."
138 | #> [19] ""
139 | #> [20] "> "
140 | process_read(handle, PIPE_STDERR)
141 | #> character(0)
142 |
The first number in the output is the value returned by process_write which is the number of characters written to standard input of the child process. The final character(0) is the output read from the standard error stream.
143 |
Next, we create a new variable in child’s session. Please notice the new-line character at the end of the command. It triggers the child process to process its input.
144 |
process_write(handle, 'n <- 10\n')
145 | #> [1] 8
146 | process_read(handle, PIPE_STDOUT, timeout = 1000)
147 | #> [1] "n <- 10" "> "
148 | process_read(handle, PIPE_STDERR)
149 | #> character(0)
150 |
Now it’s time to use this variable in a function call:
151 |
process_write(handle, 'rnorm(n)\n')
152 | #> [1] 9
153 | process_read(handle, PIPE_STDOUT, timeout = 1000)
154 | #> [1] "rnorm(n)"
155 | #> [2] " [1] -0.7766305 -0.5055096 -0.1805872 0.6010226 -0.2043694 0.7233065"
156 | #> [3] " [7] 0.9965769 0.8596935 -0.8944460 -0.9790457"
157 | #> [4] "> "
158 | process_read(handle, PIPE_STDERR)
159 | #> character(0)
160 |
Finally, we exit the child process:
161 |
process_write(handle, 'q(save = "no")\n')
162 | #> [1] 15
163 | process_read(handle, PIPE_STDOUT, timeout = 1000)
164 | #> [1] "q(save = \"no\")"
165 | process_read(handle, PIPE_STDERR)
166 | #> character(0)
167 |
The last thing is making sure that the child process is no longer alive:
168 |
process_state(handle)
169 | #> [1] "exited"
170 | process_return_code(handle)
171 | #> [1] 0
172 |
Of course there is little value in running a child R process since there are multiple other tools that let you do that, like parallel, Rserve and opencpu to name just a few. However, it’s quite easy to imagine how running a remote shell in this manner enables new ways of interacting with the environment. Consider running a local shell:
173 |
shell_binary <- function () {
174 | ifelse (tolower(.Platform$OS.type) == "windows",
175 | "C:/Windows/System32/cmd.exe", "/bin/sh")
176 | }
177 |
178 | handle <- spawn_process(shell_binary())
179 | print(handle)
180 | #> Process Handle
181 | #> command : /bin/dash
182 | #> system id : 14231
183 | #> state : running
184 |
Now we can interact with the shell sub-process. We send a request to list the current directory, then give it a moment to process the command and produce the output (and maybe finish its start-up, too). Finally, we check its output streams.
185 |
process_write(handle, "ls\n")
186 | #> [1] 3
187 | Sys.sleep(1)
188 | process_read(handle, PIPE_STDOUT)
189 | #> [1] "intro.Rmd"
190 | process_read(handle, PIPE_STDERR)
191 | #> character(0)
192 |
194 |
Advanced techniques
195 |
196 |
Terminating a child process
197 |
If the child process needs to be terminated one can choose to:
198 |
199 | - send a command on the standard input with
process_write()
200 | - send the termination signal,
SIGTERM (Linux, Windows)
201 | - send the kill signal,
SIGKILL (Linux only)
202 |
203 |
Assume the child R process is hung and there is no way to stop it gracefully. process_wait(handle, 1000) waits for 1 second (1000 milliseconds) for the child process to exit. It then returns NA and process_terminate() gives R a chance to exit graceully. Finally, process_kill() forces it to exit.
204 |
sub_command <- "library(subprocess);subprocess:::signal(15,'ignore');Sys.sleep(1000)"
205 | handle <- spawn_process(R_binary(), c('--slave', '-e', sub_command))
206 | Sys.sleep(1)
207 |
208 | # process is hung
209 | process_wait(handle, 1000)
210 | #> [1] 1
211 | process_state(handle)
212 | #> [1] "exited"
213 |
214 | # ask nicely to exit; will be ignored in Linux but not in Windows
215 | process_terminate(handle)
216 | #> [1] TRUE
217 | process_wait(handle, 1000)
218 | #> [1] 1
219 | process_state(handle)
220 | #> [1] "exited"
221 |
222 | # forced exit; in Windows the same as the previous call to process_terminate()
223 | process_kill(handle)
224 | #> [1] TRUE
225 | process_wait(handle, 1000)
226 | #> [1] 1
227 | process_state(handle)
228 | #> [1] "exited"
229 |
We see that the child process remains running until it receives the SIGKILL signal. The final return code (exit status) is the number of the signal that caused the child process to exit.
230 |
231 |
232 |
Sending a signal to the child process
233 |
The last topic we want to cover here is sending an arbitrary signal to the child process. Signals can be listed by looking at the signals variable present in the package. It is constructed automatically when the package is loaded and its value on Linux is different than on Windows. In the example below we see the first three elements of the Linux list of signals.
234 |
length(signals)
235 | #> [1] 19
236 | signals[1:3]
237 | #> $SIGHUP
238 | #> [1] 1
239 | #>
240 | #> $SIGINT
241 | #> [1] 2
242 | #>
243 | #> $SIGQUIT
244 | #> [1] 3
245 |
All possible signal identifiers are supported directly from the subprocess package. Signals not supported on the current platform are set to NA and the rest have their OS-specific numbers as their values.
246 |
ls(pattern = 'SIG', envir = asNamespace('subprocess'))
247 | #> [1] "SIGABRT" "SIGALRM" "SIGCHLD" "SIGCONT" "SIGFPE" "SIGHUP" "SIGILL"
248 | #> [8] "SIGINT" "SIGKILL" "SIGPIPE" "SIGQUIT" "SIGSEGV" "SIGSTOP" "SIGTERM"
249 | #> [15] "SIGTSTP" "SIGTTIN" "SIGTTOU" "SIGUSR1" "SIGUSR2"
250 |
Now we can create a new child process and send an arbitrary using its handle.
251 |
handle <- spawn_process(R_binary, '--slave')
252 |
253 | process_send_signal(handle, SIGUSR1)
254 |
255 |