├── DESCRIPTION ├── NAMESPACE ├── R └── mutable.R ├── README.md ├── man └── mutable.Rd ├── src └── mutable.c └── vignettes └── mutable.Rmd /DESCRIPTION: -------------------------------------------------------------------------------- 1 | Package: mutable 2 | Title: Mutable Vectors 3 | Version: 0.0-0 4 | Author: Luke Tierney 5 | Description: An implementation of mutable vectors using the ALTREP framework. 6 | Maintainer: Luke Tierney 7 | License: GPL-2 | GPL-3 8 | ByteCompile: yes 9 | Depends: R (>= 3.5) 10 | VignetteBuilder: knitr 11 | -------------------------------------------------------------------------------- /NAMESPACE: -------------------------------------------------------------------------------- 1 | # Refer to all C routines by their name prefixed by C_ 2 | useDynLib(mutable, .registration = TRUE, .fixes = "C_") 3 | 4 | export(mutable, is.mutable) 5 | -------------------------------------------------------------------------------- /R/mutable.R: -------------------------------------------------------------------------------- 1 | # Copyright 2018 R Core Team 2 | # Distributed under GPL 2 or 3 3 | 4 | mutable <- function(type = "double", length = 0L) { 5 | if (length > 0) 6 | .Call(C_make_mutable, vector(type, length)) 7 | else 8 | vector(type, 0L) 9 | } 10 | 11 | is.mutable <- function(x) 12 | .Call(C_is_mutable, x) 13 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # The `mutable` Package 2 | 3 | This package explores the possibility of using the `ALTREP` framework 4 | to provide mutable vectors in R that behave in a reasonable way. 5 | There are some major issues with this that might be resolvable, but 6 | for the moment this seems to be problematic. More detailed discussion 7 | of the issues is available in a vignette. 8 | -------------------------------------------------------------------------------- /man/mutable.Rd: -------------------------------------------------------------------------------- 1 | % Copyright 2018 R Core Team 2 | % Distributed under GPL 2 or 3 3 | 4 | \name{mutable} 5 | \alias{mutable} 6 | \alias{is.mutable} 7 | \title{Mutable Vectors} 8 | \description{ 9 | Create and test for mutable vectors. 10 | } 11 | \usage{ 12 | mutable(type = "double", length = 0L) 13 | is.mutable(x) 14 | } 15 | %- maybe also 'usage' for other objects documented here. 16 | \arguments{ 17 | \item{type}{character string naming the atomic type.} 18 | \item{length}{non-negative integer specifying the desired length.} 19 | \item{x}{an R object.} 20 | } 21 | \details{ 22 | Mutable vectors allow pass by reference semantics. THey should be used 23 | with care. 24 | } 25 | \value{ 26 | \code{mutable} returns a vector of the specified type with mutable 27 | data. Numeric vector elements are initialized to zero. 28 | 29 | \code{is.mutable} returns \code{TRUE} if its argument is a mutable 30 | vector and \code{FALSE} if it is not. 31 | } 32 | %%\references{ 33 | %% ~put references to the literature/web site here ~ 34 | %%} 35 | \author{ 36 | Luke Tierney 37 | } 38 | 39 | \examples{ 40 | x <- mutable("double", 5) 41 | x[] <- 1:5 42 | y <- x 43 | x[1] <- 6 44 | y[1] 45 | is.mutable(x) 46 | is.mutable(y) 47 | is.mutable(1:5) 48 | } 49 | \keyword{file} 50 | -------------------------------------------------------------------------------- /src/mutable.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2018 The R Core Team 3 | * Distributed under GPL 2 or 3 4 | */ 5 | 6 | #include 7 | #include 8 | #include 9 | #include 10 | 11 | /** 12 | ** Mutable Vectors 13 | **/ 14 | 15 | /* 16 | * Mutable Vector Classes and Objects 17 | */ 18 | 19 | static R_altrep_class_t mutable_integer_class; 20 | static R_altrep_class_t mutable_real_class; 21 | 22 | /* Mutable objects are ALTREP objects with data fields 23 | 24 | data1: the data vector 25 | data2: not used 26 | */ 27 | 28 | static SEXP make_mutable(SEXP data) 29 | { 30 | R_altrep_class_t class; 31 | int type = TYPEOF(data); 32 | 33 | switch(type) { 34 | case INTSXP: 35 | class = mutable_integer_class; 36 | break; 37 | case REALSXP: 38 | class = mutable_real_class; 39 | break; 40 | default: 41 | error("mutable vectors for %s not supported yet", type2char(type)); 42 | } 43 | 44 | /* At this point 'data' may be shared, but only with other mutable 45 | vectors. */ 46 | SEXP val = R_new_altrep(class, data, R_NilValue); 47 | 48 | #ifdef PREVENT_REUSE 49 | /* The result is marked as not mutable so optimizations, in 50 | particular in arithmetic operations, that re-use unreferenced 51 | vectors don't do so with one of these. THis doesn't help with 52 | code that uses a duplicate-and-modify idiom. */ 53 | MARK_NOT_MUTABLE(val); 54 | #endif 55 | 56 | return val; 57 | } 58 | 59 | #define MUTABLE_DATA(x) R_altrep_data1(x) 60 | 61 | 62 | /* 63 | * ALTREP Methods 64 | */ 65 | 66 | static SEXP mutable_Duplicate(SEXP x, Rboolean deep) 67 | { 68 | /* return a new mutable vector with the same data */ 69 | return make_mutable(MUTABLE_DATA(x)); 70 | } 71 | 72 | Rboolean mutable_Inspect(SEXP x, int pre, int deep, int pvec, 73 | void (*inspect_subtree)(SEXP, int, int, int)) 74 | { 75 | Rprintf(" mutable %s\n", type2char(TYPEOF(x))); 76 | inspect_subtree(MUTABLE_DATA(x), pre, deep, pvec); 77 | return TRUE; 78 | } 79 | 80 | static R_xlen_t mutable_Length(SEXP x) 81 | { 82 | return XLENGTH(MUTABLE_DATA(x)); 83 | } 84 | 85 | 86 | /* 87 | * ALTVEC Methods 88 | */ 89 | 90 | static void *mutable_Dataptr(SEXP x, Rboolean writeable) 91 | { 92 | if (writeable) 93 | return DATAPTR(MUTABLE_DATA(x)); 94 | else 95 | return (void *) DATAPTR_RO(MUTABLE_DATA(x)); 96 | } 97 | 98 | static const void *mutable_Dataptr_or_null(SEXP x) 99 | { 100 | return DATAPTR_OR_NULL(MUTABLE_DATA(x)); 101 | } 102 | 103 | 104 | /* 105 | * ALTINTEGER Methods 106 | */ 107 | 108 | static int mutable_integer_Elt(SEXP x, R_xlen_t i) 109 | { 110 | return INTEGER(MUTABLE_DATA(x))[i]; 111 | } 112 | 113 | static 114 | R_xlen_t mutable_integer_Get_region(SEXP x, R_xlen_t i, R_xlen_t n, int *buf) 115 | { 116 | return INTEGER_GET_REGION(MUTABLE_DATA(x), i, n, buf); 117 | } 118 | 119 | 120 | /* 121 | * ALTREAL Methods 122 | */ 123 | 124 | static double mutable_real_Elt(SEXP x, R_xlen_t i) 125 | { 126 | return REAL(MUTABLE_DATA(x))[i]; 127 | } 128 | 129 | static 130 | R_xlen_t mutable_real_Get_region(SEXP x, R_xlen_t i, R_xlen_t n, double *buf) 131 | { 132 | return REAL_GET_REGION(MUTABLE_DATA(x), i, n, buf); 133 | } 134 | 135 | 136 | /* 137 | * Class Objects and Method Tables 138 | */ 139 | 140 | #define MUTPKG "mutable" 141 | 142 | static void InitMutableIntegerClass(DllInfo *dll) 143 | { 144 | R_altrep_class_t cls = 145 | R_make_altinteger_class("mutable_integer", MUTPKG, dll); 146 | mutable_integer_class = cls; 147 | 148 | /* override ALTREP methods */ 149 | R_set_altrep_Duplicate_method(cls, mutable_Duplicate); 150 | R_set_altrep_Inspect_method(cls, mutable_Inspect); 151 | R_set_altrep_Length_method(cls, mutable_Length); 152 | 153 | /* override ALTVEC methods */ 154 | R_set_altvec_Dataptr_method(cls, mutable_Dataptr); 155 | R_set_altvec_Dataptr_or_null_method(cls, mutable_Dataptr_or_null); 156 | 157 | /* override ALTINTEGER methods */ 158 | R_set_altinteger_Elt_method(cls, mutable_integer_Elt); 159 | R_set_altinteger_Get_region_method(cls, mutable_integer_Get_region); 160 | } 161 | 162 | static void InitMutableRealClass(DllInfo *dll) 163 | { 164 | R_altrep_class_t cls = 165 | R_make_altreal_class("mutable_real", MUTPKG, dll); 166 | mutable_real_class = cls; 167 | 168 | /* override ALTREP methods */ 169 | R_set_altrep_Duplicate_method(cls, mutable_Duplicate); 170 | R_set_altrep_Inspect_method(cls, mutable_Inspect); 171 | R_set_altrep_Length_method(cls, mutable_Length); 172 | 173 | /* override ALTVEC methods */ 174 | R_set_altvec_Dataptr_method(cls, mutable_Dataptr); 175 | R_set_altvec_Dataptr_or_null_method(cls, mutable_Dataptr_or_null); 176 | 177 | /* override ALTREAL methods */ 178 | R_set_altreal_Elt_method(cls, mutable_real_Elt); 179 | R_set_altreal_Get_region_method(cls, mutable_real_Get_region); 180 | } 181 | 182 | 183 | /* 184 | * Constructor 185 | */ 186 | 187 | SEXP do_make_mutable(SEXP x) 188 | { 189 | if (MAYBE_REFERENCED(x)) 190 | x = duplicate(x); 191 | PROTECT(x); 192 | SEXP ans = make_mutable(x); 193 | UNPROTECT(1); /* x */ 194 | return ans; 195 | } 196 | 197 | SEXP do_is_mutable(SEXP x) 198 | { 199 | switch(TYPEOF(x)) { 200 | case INTSXP: 201 | return ScalarLogical(R_altrep_inherits(x, mutable_integer_class)); 202 | case REALSXP: 203 | return ScalarLogical(R_altrep_inherits(x, mutable_real_class)); 204 | default: 205 | return ScalarLogical(FALSE); 206 | } 207 | } 208 | 209 | 210 | /* 211 | * Shared Library Initialization 212 | */ 213 | 214 | static const R_CallMethodDef CallEntries[] = { 215 | {"make_mutable", (DL_FUNC) &do_make_mutable, 1}, 216 | {"is_mutable", (DL_FUNC) &do_is_mutable, 1}, 217 | {NULL, NULL, 0} 218 | }; 219 | 220 | void R_init_mutable(DllInfo *dll) 221 | { 222 | InitMutableIntegerClass(dll); 223 | InitMutableRealClass(dll); 224 | 225 | R_registerRoutines(dll, NULL, CallEntries, NULL, NULL); 226 | } 227 | -------------------------------------------------------------------------------- /vignettes/mutable.Rmd: -------------------------------------------------------------------------------- 1 | 5 | 6 | ```{r global_options, include=FALSE} 7 | knitr::opts_chunk$set(collapse=TRUE) 8 | ``` 9 | 10 | 11 | # Mutable Vectors in R 12 | 13 | R vectors are conceptually immutable. Replacement functions create a 14 | modified copy of the original vector. So in this example the value of 15 | `y` is unchanged: 16 | 17 | ```{r} 18 | x <- 1:3 19 | y <- x 20 | x[1] <- 7 21 | y 22 | ``` 23 | ```{r, include = FALSE} 24 | stopifnot(y[1] == 1) 25 | ``` 26 | 27 | This computational model avoids many bugs that might occur with a pass 28 | by reference model, but it comes at the price of the overhead of 29 | duplicating objects before modifying them. R does try to avoid 30 | duplication when it can prove doing so is safe, but sometimes it 31 | cannot be voided. 32 | 33 | To avoid this potential overhead when initially building up an object, 34 | or to maintain state, it might be useful to have R vectors that are 35 | mutable, or have reference semantics, but otherwise behave like 36 | ordinary R vectors. S3 or S4 classes can be used to some extent to 37 | achieve this, but the resulting objects will not behave like ordinary 38 | R vectors in all respects, in particular at the `C` level. 39 | 40 | The `mutable` package was created to explore whether the `ALTREP` 41 | framework can be used to create mutable R vectors that behave 42 | properly. The answer seems to be no, at least at this point. 43 | 44 | 45 | ## The `mutable` Package 46 | 47 | The `mutable` package provides an attempt to implement of mutable 48 | vectors in R using the `ALTREP` framework. 49 | 50 | Mutable vectors are created by `mutable`: 51 | ```{r} 52 | library(mutable) 53 | (x <- mutable("double", 2)) 54 | y <- x 55 | x[1] <- 1 56 | y 57 | ``` 58 | ```{r, include = FALSE} 59 | stopifnot(x[1] == y[1]) 60 | ``` 61 | 62 | The data portions of `x` and `y` are shared, so modifying the data of 63 | `x` also modifies the data of `y`. Only the data part of the vector 64 | is shared; attribute assignments obey standard R semantics: 65 | 66 | ```{r} 67 | attr(x, "foo") <- 1 68 | attributes(y) 69 | ``` 70 | ```{r, include = FALSE} 71 | stopifnot(is.null(attributes(y))) 72 | ``` 73 | 74 | The function `is.mutable` tests whether its argument was created by 75 | `mutable` (it does not allow for wrappers at this point). 76 | 77 | ```{r} 78 | is.mutable(x) 79 | is.mutable(y) 80 | is.mutable(1:2) 81 | ``` 82 | ```{r, include = FALSE} 83 | stopifnot(is.mutable(x), is.mutable(y), ! is.mutable(1:2)) 84 | ``` 85 | 86 | Possible additions, that could be done at the R level: 87 | 88 | - Allow specifying initial data in the `mutable()` call. 89 | - Add a `freeze` or `unmutable` function. 90 | 91 | 92 | For now, only `integer` and `double` vectors are supported. Other 93 | atomic types shoud be easy to add. Support for `character` vectors is 94 | also probably straight forward. Vectors of type `list` may be more 95 | challenging as cycles need to be prevented. 96 | 97 | There is no special serialization support: A mutable vector is 98 | serialized as a standard unmutable vector. Sharing could be preserved 99 | within a serialization by using an external pointer as an 100 | intermediary. It's not clear this would be worth while. 101 | 102 | There is no support for growing or shrinking a mutable vector. 103 | Support along the lines of Common Lisp fill pointers could be 104 | added. It's not clear whether integrating this with the growable 105 | vector concept used when assigning beyond the current length of a 106 | vector sould be a good idea. 107 | 108 | At the `C` level both deep and shallow duplicating preserves data 109 | sharing. Internally the object uses a separate R vector for the shared 110 | storage, but this vector should not be accessed outside the package 111 | `C` code. 112 | 113 | 114 | ## Problems 115 | 116 | The base R code is written for the standard immutable vector 117 | model. This leads to certain coding practices that are not compatible 118 | with allowing vectors to have mutable data. Most of these are based on 119 | the assumption that an object with `NO_REFERENCES` can be modified in 120 | place r re-used. Two particular practices are: 121 | 122 | - It is common at the `C` level to `duplicate` or `shallow_duplicate` 123 | an object and then modify it. 124 | 125 | - In arithmetic operations in particular an argument with no 126 | references may be re-used for the result to avoid allocating a 127 | result vector. 128 | 129 | ### Duplicate and Modify 130 | 131 | The duplicate and modify approach is This approach is based on the 132 | assumption that `duplicate` and `shallow_duplicate` vector that have 133 | no references and can be freely modified. This approach is used in 134 | many unary operations, including unary minus: 135 | 136 | ```{r} 137 | x <- mutable("double", 2); x[] <- 1:2 138 | -x 139 | x 140 | ``` 141 | 142 | The attempt to create a mutable vector by having the `Duplicate` 143 | method share the data portion violates the `duplicate` and 144 | `shallow_duplicate` and would require modifying base R code, and 145 | probably many packages, in many places to make it work properly. 146 | 147 | Addressing this issue would require every use of the duplicate and 148 | modify idiom to be replaced by allocating a new standard vector and 149 | copying data and attributes as needed. The effort would be 150 | significant. It would also loose the ability to automatically reuse 151 | `ALTREP` objects that is currently possible, unless we added a 152 | `Can_reuse` method to determine whether this is OK or not. 153 | 154 | 155 | ### Value Re-Use 156 | 157 | Without re-use an arithmetic operation with a mutable vector would 158 | produce a standard vector result: 159 | 160 | ```{r} 161 | x <- mutable("double", 2) 162 | is.mutable(x + 1) 163 | ``` 164 | 165 | But when the mutable vector has no references it maybe re-used as the result: 166 | 167 | ```{r} 168 | is.mutable(mutable("double", 2) + 1) 169 | ``` 170 | 171 | Reusing an unreferenced mutable vector as a result can allow 172 | computations to unexpectedly produce mutable results, thus risking the 173 | bugs that the basic R design is intended to prevent. 174 | 175 | Marking the result produced by `mutable` as having references could 176 | avoid this issue but not the duplicate-and-modify issue. It would also 177 | force duplication of the `ALTREP` cell for any update operation. 178 | 179 | 180 | 186 | --------------------------------------------------------------------------------