├── .gitignore ├── README.md ├── examples ├── example1.py ├── example2.py └── example3.py ├── pyterator ├── __init__.py └── pyterator.py ├── requirements.txt ├── setup.py └── tests ├── __init__.py ├── test_pyterator.py └── test_transforms.py /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__ 2 | .DS_Store 3 | .pytest_cache 4 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Pyterator 2 | 3 | Write fluent functional programming idioms in Python. 4 | 5 | - Chain operations like `map`, `reduce`, `filter_map` 6 | - Lazy evaluation 7 | 8 | Readable transformation functions, as opposed to Lisp-ish prefix notation-esque map filter functions. 9 | 10 | --- 11 | 12 | ## Contents 13 | 14 | - [Installation](#installation) 15 | - [Quick start](#quick-start) 16 | - [How it works](#how-it-works) 17 | - [Motivation](#motivation) 18 | - [Design](#design) 19 | - [How is it different?](#how-it-works) 20 | - [Who is this for (and not for)?](#anyone) 21 | - [Design](#design) 22 | - [Examples](#examples) 23 | 24 | ## Installation 25 | 26 | ```bash 27 | pip install git+https://github.com/remykarem/pyterator.git#egg=pyterator 28 | ``` 29 | 30 | ## Quick start 31 | 32 | ```python 33 | >>> from pyterate import iterate 34 | ``` 35 | 36 | ```python 37 | >>> text = ["hello", "world"] 38 | >>> iterate(text).map(str.upper).to_list() 39 | ['HELLO', 'WORLD'] 40 | ``` 41 | 42 | Chain operations 43 | 44 | ```python 45 | >>> text = ["hello", "my", "love"] 46 | >>> ( 47 | ... iterate(text) 48 | ... .filterfalse(lambda x: x in ["a", "my"]) 49 | ... .map(str.upper) 50 | ... .map(lambda x: x+"!") 51 | ... .to_list() 52 | ... ) 53 | ['HELLO!', 'LOVE!'] 54 | ``` 55 | 56 | ## Motivation 57 | 58 | Using `map`, `reduce` in Python forces you to write in prefix notation-esque which makes it hard to read. 59 | 60 | For a transformation pipeline: 61 | 62 | ```python 63 | [1, 2, 3, 4] -> [2, 3, 4, 5] -> [3, 5] -> 15 64 | ``` 65 | 66 | Python: 67 | 68 | ```python 69 | reduce(lambda x,y: x * y, 70 | filter(lambda x: x % 2, 71 | map(lambda x: x+1, [1, 2, 3, 4])), 1) 72 | ``` 73 | 74 | which looks similar to Common Lisp 75 | 76 | ```lisp 77 | (reduce #'* 78 | (remove-if #'evenp 79 | (mapcar (lambda (x) (1+ x)) '(1 2 3 4)))) 80 | ``` 81 | 82 | and Haskell 83 | 84 | ```haskell 85 | foldl (*) 1 86 | (filter odd 87 | (map (\x -> x+1) [1, 2, 3, 4])) 88 | ``` 89 | 90 | which are languages where prefix notation is their natural syntax. 91 | 92 | List comprehensions, while idiomatic and commonplace among developers, can be hard to read at times. 93 | 94 | ## Design 95 | 96 | This design is largely influenced by modern languages that implement functional programming idioms like Rust, Kotlin, Scala and JavaScript. The Apache Spark framework, which is written in Scala, largely exposes functional programming idioms in the Python APIs. 97 | 98 | We want the subject of the chain of transformations to be the data itself, then call the operations in succession: 99 | 100 | ```python 101 | ( 102 | [1,2,3,4] 103 | .map(...) 104 | .filter(...) 105 | .reduce(...) 106 | ) 107 | ``` 108 | 109 | and 110 | 111 | ```python 112 | ( 113 | iter([1,2,3,4]) 114 | .map(...) 115 | .filter(...) 116 | .reduce(...) 117 | ) 118 | ``` 119 | 120 | Since Python's iterator does not have methods `map`, `reduce`, we implemented our own `iterate`, which is similar to Python's builtin `iter`, so that client code can easily switch it out. 121 | 122 | ```python 123 | ( 124 | iterate([1,2,3,4]) 125 | .map(...) 126 | .filter(...) 127 | .reduce(...) 128 | ) 129 | ``` 130 | 131 | iterator, 132 | 133 | It is also a builder function, which returns a `_Pyterator` instance that implements `__next__`. 134 | 135 | ## How it works 136 | 137 | Lazy. Reduce operations and to_list() operations will 'materialise' your transformations. 138 | 139 | ## Examples 140 | 141 | ### Example 1: Square 142 | 143 | ```txt 144 | [1, 2, 3, 4] -> [1, 4, 9, 16] 145 | ``` 146 | 147 | ```python 148 | >>> from pyterator import iterate 149 | >>> nums = [1, 2, 3, 4] 150 | ``` 151 | 152 | Pyterator 153 | 154 | ```python 155 | >>> iterate(nums).map(lambda x: x**2).to_list() 156 | ``` 157 | 158 | List comprehension 159 | 160 | ```python 161 | >>> [x**2 for x in nums] 162 | ``` 163 | 164 | Map reduce 165 | 166 | ```python 167 | >>> list(map(lambda x: x**2, iter(nums))) 168 | ``` 169 | 170 | ### Example 2: Filter 171 | 172 | Pyterator 173 | 174 | ```python 175 | >>> iterate(nums).filter(lambda x: x > 3).to_list() 176 | ``` 177 | 178 | List comprehension 179 | 180 | ```python 181 | >>> [x for x in nums if x > 3] 182 | ``` 183 | 184 | ### Flat map 185 | 186 | ```python 187 | [ 188 | "peter piper", 189 | "picked a peck", -> 190 | "of pickled pepper", 191 | ] 192 | ``` 193 | 194 | List comprehension 195 | 196 | ```python 197 | >>> [word for text in texts for word in text.split()] 198 | ``` 199 | 200 | Pyterator 201 | 202 | ```python 203 | >>> iterate(texts).flat_map(str.split).to_list() 204 | ``` 205 | 206 | ### Multiple transformations 207 | 208 | ```python 209 | >>> from pyterator import iterate 210 | >>> stopwords = {"of", "a"} 211 | >>> texts = [ 212 | "peter piper Picked a peck ", 213 | "of Pickled pePper", 214 | "\na peck of pickled pepper peter piper picked", 215 | ] 216 | ``` 217 | 218 | List comprehension 219 | 220 | ```python 221 | >>> words = [ 222 | word for text in texts 223 | for word in text.lower().strip().split() 224 | if word not in stopwords] 225 | >>> set(words) 226 | {'peck', 'pepper', 'peter', 'picked', 'pickled', 'piper'} 227 | ``` 228 | 229 | Pyterator 230 | 231 | ```python 232 | >>> ( 233 | ... iterate(texts) 234 | ... .map(str.strip) 235 | ... .map(str.lower) 236 | ... .flat_map(str.split) 237 | ... .filter(lambda word: word not in stopwords) 238 | ... .to_set() 239 | ... ) 240 | {'peck', 'pepper', 'peter', 'picked', 'pickled', 'piper'} 241 | ``` 242 | 243 | ## Inspired by 244 | 245 | https://doc.rust-lang.org/std/iter/trait.Iterator.html 246 | 247 | ## Gotchas 248 | 249 | These gotchas pertain to mutability of the collections 250 | 251 | ## What this is not for 252 | 253 | Vectorised operations - use NumPy or other equivalent 254 | 255 | ## API 256 | 257 | common 258 | 259 | - `map` 260 | - `enumerate` 261 | - `filter` 262 | - `for_each` 263 | - `filterfalse` 264 | - `filter_map` 265 | - `starmap` 266 | - `starfilter` 267 | - `star_map_filter` 268 | 269 | order 270 | 271 | - `reverse` 272 | 273 | dimension change 274 | 275 | - `partition` 276 | - `flat_map` 277 | - `star_flat_map` 278 | - `chunked` 279 | - `flatten` 280 | - `zip` 281 | - `chain` 282 | 283 | positional 284 | 285 | - `skip` 286 | - `first` 287 | - `nth` 288 | - `take` 289 | 290 | collect 291 | 292 | - `to_list` 293 | - `to_set` 294 | - `to_dict` 295 | - `groupby` 296 | 297 | reduce 298 | 299 | - `reduce` 300 | - `all` 301 | - `any` 302 | - `min` 303 | - `max` 304 | - `sum` 305 | - `prod` 306 | - `join` 307 | - `sample` 308 | 309 | ## Similar libraries 310 | 311 | Note that these libraries focus on fluent method chaining. 312 | 313 | - [PyFunctional](https://github.com/EntilZha/PyFunctional) 314 | - [fluent](https://github.com/dwt/fluent) 315 | - [Simple Smart Pipe](https://github.com/sspipe/sspipe) 316 | - [pyxtension](https://github.com/asuiu/pyxtension) 317 | - [pyiter](https://github.com/mokeyish/pyiter) 318 | -------------------------------------------------------------------------------- /examples/example1.py: -------------------------------------------------------------------------------- 1 | """ 2 | Example taken from 3 | https://realpython.com/introduction-to-python-generators/ 4 | """ 5 | from pyterator import iterate 6 | 7 | 8 | text = """permalink,company,numEmps,category,city,state,fundedDate,raisedAmt,raisedCurrency,round 9 | digg,Digg,60,web,San Francisco,CA,1-Dec-06,8500000,USD,b 10 | digg,Digg,60,web,San Francisco,CA,1-Oct-05,2800000,USD,a 11 | facebook,Facebook,450,web,Palo Alto,CA,1-Sep-04,500000,USD,angel 12 | facebook,Facebook,450,web,Palo Alto,CA,1-May-05,12700000,USD,a 13 | photobucket,Photobucket,60,web,Palo Alto,CA,1-Mar-05,3000000,USD,a""" 14 | 15 | data = ( 16 | iterate(text.split("\n")) 17 | .map(lambda s: s.split(",")) 18 | .to_gen() 19 | ) 20 | 21 | header = next(data) 22 | 23 | ans = ( 24 | iterate(data) 25 | .flat_map(lambda row: zip(header, row)) 26 | .groupby(lambda x: x[0], lambda x: x[1]) 27 | ) 28 | 29 | assert ans['permalink'] == ['digg', 30 | 'digg', 31 | 'facebook', 32 | 'facebook', 33 | 'photobucket'] 34 | -------------------------------------------------------------------------------- /examples/example2.py: -------------------------------------------------------------------------------- 1 | from pyterator import iterate 2 | 3 | cipher = "rovzwo" 4 | clear = ( 5 | iterate(cipher) 6 | .map(ord) 7 | .map("-", 10) 8 | .map(chr) 9 | .join() 10 | ) 11 | assert clear == "helpme" 12 | -------------------------------------------------------------------------------- /examples/example3.py: -------------------------------------------------------------------------------- 1 | from typing import Optional 2 | from pyterator import iterate 3 | 4 | text = """Survived,Pclass,Name,Sex,Age,Siblings/Spouses Aboard,Parents/Children Aboard,Fare 5 | 0,3,Mr+ Owen Harris Braund,male,22,1,0,7.25 6 | 1,3,Miss. Laina Heikkinen,female,26,0,0,7.925 7 | 1,1,Mrs. Jacques Heath (Lily May Peel) Futrelle,female,35,1,0,53.1 8 | 0 9 | 0,3,Mr. James Moran,male,27,0,0,8.4583 10 | 0,1,Mr. Timothy J McCarthy,male,54,0,0,51.8625 11 | 0,3,Master. Gosta Leonard Palsson,male,2,3,1,21.075 12 | 1,3,Mrs. Oscar W (Elisabeth Vilhelmina Berg) Johnson,female,27,0,2,11.1333 13 | 1,1,Miss+ Elizabeth Bonnell,female,58,0,0,26.55 14 | 0,3,Mr. William Henry Saundercock,male,20,0,0,8.05""" 15 | 16 | 17 | def get_name_from_line(line: str) -> Optional[str]: 18 | # We can use the walrus operator here 19 | name = line.split(",")[2:3] 20 | if name: 21 | return name[0] 22 | else: 23 | return None 24 | 25 | def get_name_split(name: str) -> Optional[list]: 26 | # We can use the walrus operator here 27 | split = name.split(". ") 28 | if len(split) == 2: 29 | return split 30 | else: 31 | return None 32 | 33 | title_names: dict = ( 34 | iterate(text.split("\n")) 35 | .filter_map(get_name_from_line) 36 | .filter_map(get_name_split) 37 | .groupby(lambda name: name[0], lambda name: name[1]) 38 | ) 39 | -------------------------------------------------------------------------------- /pyterator/__init__.py: -------------------------------------------------------------------------------- 1 | from pyterator.pyterator import iterate, tee 2 | 3 | __ALL__ = ['iterate', 'tee'] 4 | -------------------------------------------------------------------------------- /pyterator/pyterator.py: -------------------------------------------------------------------------------- 1 | from __future__ import annotations 2 | 3 | from functools import reduce 4 | from collections import Counter 5 | from collections.abc import Iterable 6 | from typing import Any, Callable, Generator, Iterator, List, Tuple, Optional 7 | from itertools import chain, filterfalse, islice, product, starmap 8 | from itertools import tee as _tee 9 | 10 | from more_itertools import ( 11 | chunked, map_reduce, sample, partition, islice_extended, 12 | split_at, windowed 13 | ) 14 | 15 | 16 | def iterate(iterable: Iterable[Any]) -> _Pyterator: 17 | """Similar to the builtin `iter` object""" 18 | return _Pyterator(iterable) 19 | 20 | 21 | def tee(iterable: Iterable[Any], n: int = 2) -> List[_Pyterator]: 22 | """Similar to `itertools.tee`""" 23 | iters = _tee(iterable, n) 24 | return [_Pyterator(it) for it in iters] 25 | 26 | 27 | class _Pyterator: 28 | 29 | def __init__(self, iterable: Iterable): 30 | self.__iterator = iter(iterable) 31 | 32 | def __repr__(self) -> str: 33 | return f"" 34 | 35 | def __iter__(self) -> Iterator: 36 | return self.__iterator 37 | 38 | def __next__(self) -> Any: 39 | return next(self.__iterator) 40 | 41 | def reverse(self) -> _Pyterator: 42 | """Reverses the order of the elements in the underlying iterator 43 | 44 | Returns: 45 | _Pyterator: _Pyterator object 46 | """ 47 | self.__iterator = islice_extended(self.__iterator, -1, None, -1) 48 | return self 49 | 50 | def map(self, fn: Callable) -> _Pyterator: 51 | """Similar to the builtin `map` object. Returns an iterator 52 | yielding the results of applying the function to the items of 53 | iterable. 54 | 55 | Args: 56 | fn (Callable): Function to apply to each item 57 | 58 | Returns: 59 | _Pyterator: _Pyterator object 60 | """ 61 | self.__iterator = map(fn, self.__iterator) 62 | return self 63 | 64 | def starmap(self, fn: Callable) -> _Pyterator: 65 | """Similar to the `itertools.starmap` object. Applies function to 66 | arguments obtained from every item in the underlying iterator. 67 | 68 | Args: 69 | fn (Callable): Function to apply to each item 70 | 71 | Returns: 72 | _Pyterator: _Pyterator object 73 | """ 74 | self.__iterator = starmap(fn, self.__iterator) 75 | return self 76 | 77 | def filter(self, predicate_fn: Callable) -> _Pyterator: 78 | """Similar to the builtin `filter`. Yields items of iterable for which 79 | function(item) is true. If function is None, return items that are true. 80 | 81 | Args: 82 | predicate_fn (Callable): Predicate 83 | 84 | Returns: 85 | _Pyterator: Pyterator object 86 | """ 87 | self.__iterator = filter(predicate_fn, self.__iterator) 88 | return self 89 | 90 | def starfilter(self, fn: Callable) -> _Pyterator: 91 | """Similar to the builtin `filter`. Yields items of iterable for which 92 | function(*item) is true. If function is None, return items that are true. 93 | 94 | Args: 95 | fn (Callable): Predicate 96 | 97 | Returns: 98 | _Pyterator: _Pyterator object 99 | """ 100 | self.__iterator = filter(lambda args: fn(*args), self.__iterator) 101 | return self 102 | 103 | def filterfalse(self, fn: Callable) -> _Pyterator: 104 | self.__iterator = filterfalse(fn, self.__iterator) 105 | return self 106 | 107 | def filter_map(self, fn: Callable) -> _Pyterator: 108 | """ 109 | Yields items of iterable for which fn(item) is truthy. 110 | 111 | Args: 112 | fn (Callable): Function to apply to each item 113 | 114 | Returns: 115 | _Pyterator: _Pyterator object 116 | """ 117 | self.map(fn) 118 | self.__iterator = filter(lambda x: x, self.__iterator) 119 | return self 120 | 121 | def star_filter_map(self, fn: Callable) -> _Pyterator: 122 | self.starmap(fn) 123 | self.__iterator = filter(lambda x: x, self.__iterator) 124 | return self 125 | 126 | def for_each(self, fn: Callable) -> None: 127 | for x in self.__iterator: 128 | fn(x) 129 | 130 | def enumerate(self, start: int = 0) -> _Pyterator: 131 | """Similar to the builtin `enumerate` object. Yields a tuple containing 132 | an index and a value from the iterable. 133 | 134 | Args: 135 | start (int, optional): Start index. Defaults to 0. 136 | 137 | Returns: 138 | _Pyterator: Pyterator object 139 | """ 140 | self.__iterator = enumerate(self.__iterator, start) 141 | return self 142 | 143 | ### Dimensional ### 144 | 145 | def chain(self, *iterables: Iterable) -> _Pyterator: 146 | self.__iterator = chain(self.__iterator, *iterables) 147 | return self 148 | 149 | def zip(self, iterable: Iterable) -> _Pyterator: 150 | self.__iterator = zip(self.__iterator, iterable) 151 | return self 152 | 153 | def flat_map(self, fn: Callable) -> _Pyterator: 154 | return self.map(fn).flatten() 155 | 156 | def star_flat_map(self, fn: Callable) -> _Pyterator: 157 | return self.starmap(fn).flatten() 158 | 159 | def flatten(self): 160 | self.__iterator = chain.from_iterable(self.__iterator) 161 | return self 162 | 163 | def partition(self, predicate_fn: Callable) -> Tuple[_Pyterator, _Pyterator]: 164 | items_false, items_true = partition(predicate_fn, self.__iterator) 165 | return _Pyterator(items_true), _Pyterator(items_false) 166 | 167 | def chunked(self, n: int) -> _Pyterator: 168 | self.__iterator = chunked(self.__iterator, n) 169 | return self 170 | 171 | def split_at(self, predicate_fn: Callable) -> Generator[_Pyterator]: 172 | for it in split_at(self.__iterator, predicate_fn): 173 | yield _Pyterator(it) 174 | 175 | def product(self, *iterables: Iterable) -> _Pyterator: 176 | """Cartesian product""" 177 | self.__iterator = product(self.__iterator, *iterables) 178 | return self 179 | 180 | def windowed(self, window_size: int, step: int, fillvalue: Optional[Any] = None): 181 | self.__iterator = windowed(self.__iterator, window_size, step=step, fillvalue=fillvalue) 182 | return self 183 | 184 | def unique_everseen(self, key=None) -> _Pyterator: 185 | """ 186 | Yield unique elements, preserving order. Remember all elements ever seen. 187 | """ 188 | return _Pyterator(_unique_everseen(self.__iterator, key)) 189 | 190 | # Positional 191 | 192 | def skip(self, n: int) -> _Pyterator: 193 | """Creates an iterator that skips the first n elements. 194 | 195 | Args: 196 | n (int): No. of elements to skip 197 | 198 | Returns: 199 | _Pyterator: self 200 | """ 201 | self.__iterator = islice(self.__iterator, n, None, 1) 202 | return self 203 | 204 | def first(self, default: Optional[Any] = None) -> Any: 205 | """Returns the first element of the iterator. 206 | 207 | Args: 208 | default (Any, optional): Value to return if there is no element 209 | in the iterator. Defaults to None. 210 | 211 | Returns: 212 | Any: First element of the iterator 213 | """ 214 | return next(self.__iterator, default) 215 | 216 | def nth(self, n: int, default: Optional[Any] = None) -> Any: 217 | """Returns the nth element of the iterator. 218 | If n is larger than the length of the iterator, return default. 219 | The underlying iterator is advanced by n elements. 220 | 221 | Args: 222 | n (int): Index of element to return 223 | default (Any, optional): Value to return is n is larger 224 | than the length of iterator. Defaults to None. 225 | 226 | Returns: 227 | Any: nth element of the iterator 228 | """ 229 | return next(islice(self.__iterator, n, None), default) 230 | 231 | def islice(self, *args: int) -> Iterator[Any]: 232 | """Returns the next n elements. Similar to itertools.islice. 233 | The underlying iterator will be advanced by n elements. 234 | 235 | >>> iterate([9, 8, 7]).islice(2) 236 | [9, 8] 237 | >>> iterate([9, 8, 7]).islice(2, 3) 238 | [7] 239 | >>> iterate([9, 8, 6, 5, 7, 4]).islice(1, 6, 2) 240 | [8, 5, 4] 241 | 242 | Args: 243 | n (int): No. of elements to return 244 | 245 | Returns: 246 | Iterator: Iterator object 247 | """ 248 | return islice(self.__iterator, *args) 249 | 250 | def take(self, n: int) -> _Pyterator: 251 | """ 252 | Yields the first n elements, or fewer if the underlying iterator 253 | ends sooner. 254 | 255 | Args: 256 | n (int): No. of elements to take 257 | 258 | Returns: 259 | _Pyterator: self 260 | """ 261 | self.__iterator = islice(self.__iterator, n) 262 | return self 263 | 264 | ### Collection methods ### 265 | 266 | def to_list(self) -> list: 267 | """ 268 | Returns a list from the iterable's elements. 269 | 270 | Returns: 271 | list: List of elements 272 | """ 273 | return list(self.__iterator) 274 | 275 | def to_tuple(self) -> tuple: 276 | """ 277 | Returns a tuple from the iterable's elements. 278 | 279 | Returns: 280 | tuple: Tuple of elements 281 | """ 282 | return tuple(self.__iterator) 283 | 284 | def to_set(self) -> set: 285 | """ 286 | Returns a set from the iterable's elements. 287 | 288 | Returns: 289 | set: Set of elements 290 | """ 291 | return set(self.__iterator) 292 | 293 | def to_dict(self) -> dict: 294 | """ 295 | Returns a dictionary from the iterable's elements. The keys are the elements. 296 | """ 297 | return dict(self.__iterator) 298 | 299 | def to_counter(self) -> dict: 300 | """ 301 | Returns a count from the iterable's elements. The keys are the elements. 302 | """ 303 | return Counter(self.__iterator) 304 | 305 | def groupby(self, *args) -> dict: 306 | return map_reduce(self.__iterator, *args) 307 | 308 | ### Reduce functions ### 309 | 310 | def sample(self, k: int = 1) -> Any: 311 | """Return a *k*-length list of elements chosen (without replacement) 312 | from the *iterable*. Like :func:`random.sample`, but works on iterables 313 | of unknown length.""" 314 | return sample(self.__iterator, k) 315 | 316 | def reduce(self, fn: Callable, initial: Optional[Any] = None) -> Any: 317 | """ 318 | Similar to `functools.reduce`. Apply a function of two arguments 319 | cumulatively to the items of a sequence from left to right. 320 | 321 | Args: 322 | fn (Callable): function to apply to every 2 items 323 | 324 | Returns: 325 | Any: result of the reduction 326 | """ 327 | if initial: 328 | return reduce(fn, self.__iterator, initial) 329 | else: 330 | return reduce(fn, self.__iterator) 331 | 332 | def all(self) -> bool: 333 | """Return True if bool(x) is True for all values x in the iterable. 334 | 335 | If the iterable is empty, return True. 336 | 337 | Returns: 338 | bool: True if bool(x) is True for all values x in the iterable. 339 | """ 340 | return all(self.__iterator) 341 | 342 | def any(self) -> bool: 343 | """Return True if bool(x) is True for any x in the iterable. 344 | 345 | If the iterable is empty, return False. 346 | 347 | Returns: 348 | bool: True if bool(x) is True for any x in the iterable. 349 | """ 350 | return any(self.__iterator) 351 | 352 | def max(self) -> Any: 353 | """Gets the max of all elements in the iterable. 354 | 355 | Returns: 356 | Any: Max of all elements in the iterable. 357 | """ 358 | return max(self.__iterator) 359 | 360 | def min(self) -> Any: 361 | """Gets the min of all elements in the iterable. 362 | 363 | Returns: 364 | Any: Min of all elements in the iterable. 365 | """ 366 | return min(self.__iterator) 367 | 368 | def sum(self) -> Any: 369 | """Gets the sum of all elements in the iterable. 370 | 371 | Returns: 372 | Any: Sum of all elements in the iterable. 373 | """ 374 | return sum(self.__iterator) 375 | 376 | def prod(self) -> Any: 377 | """Gets the product of all elements in the iterable. 378 | 379 | Returns: 380 | Any: Product of all elements in the iterable. 381 | """ 382 | return reduce(lambda x, y: x * y, self.__iterator) 383 | 384 | def join(self, sep: str = "") -> str: 385 | """Concatenate any number of strings. 386 | 387 | Args: 388 | sep (str, optional): Separator. Defaults to "". 389 | 390 | Returns: 391 | str: Concatenated string 392 | """ 393 | return sep.join(self.__iterator) 394 | 395 | ### Debug ### 396 | 397 | def debug(self) -> None: 398 | """ 399 | Creates a REPL-like interface for debugging. Best used in an IPython-like environment. 400 | Not for use in production. 401 | 402 | Current iterator will not be consumed because an independent iterator is returned. 403 | """ 404 | # Create an independent iterator for debugging 405 | self.__iterator, iterator = _tee(self.__iterator) 406 | 407 | print("Hit ENTER for next value, q+ENTER to quit.") 408 | while True: 409 | v = input() 410 | if v.lower() == "q": 411 | break 412 | try: 413 | print(f"> {next(iterator)}", end=" ") 414 | except StopIteration: 415 | print("End of iterator") 416 | break 417 | 418 | def _unique_everseen(iterable, key=None): 419 | """From https://docs.python.org/3/library/itertools.html 420 | List unique elements, preserving order. Remember all elements ever seen. 421 | 422 | >>> _unique_everseen('AAAABBBCCDAABBB') --> A B C D 423 | >>> _unique_everseen('ABBCcAD', str.lower) --> A B C D 424 | """ 425 | seen = set() 426 | seen_add = seen.add 427 | if key is None: 428 | for element in filterfalse(seen.__contains__, iterable): 429 | seen_add(element) 430 | yield element 431 | else: 432 | for element in iterable: 433 | k = key(element) 434 | if k not in seen: 435 | seen_add(k) 436 | yield element 437 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | more-itertools==8.5.0 2 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | import os 2 | from setuptools import setup, find_packages 3 | 4 | setup( 5 | name="Pyterator", 6 | description="Pyterator helps you write fluent functional programming " 7 | "idioms in Python via chaining", 8 | version="0.0.1a2", 9 | url="https://github.com/remykarem/pyterator", 10 | author="Raimi bin Karim", 11 | author_email="raimi.bkarim@gmail.com", 12 | maintainer="Raimi bin Karim", 13 | maintainer_email="raimi.bkarim@gmail.com", 14 | license="MIT", 15 | keywords="functional pipeline data collection chain", 16 | packages=find_packages(exclude=["docs", "tests"]), 17 | long_description=(open("README.md").read() if os.path.exists("README.md") 18 | else ""), 19 | install_requires=["more-itertools>=8.11.0"], 20 | python_requires=">=3.5", 21 | classifiers=[ 22 | "Programming Language :: Python", 23 | "Programming Language :: Python :: 3", 24 | "Programming Language :: Python :: 3.5", 25 | "Programming Language :: Python :: 3.6", 26 | "Programming Language :: Python :: 3.7", 27 | "Programming Language :: Python :: 3.8", 28 | "Programming Language :: Python :: 3.9", 29 | "Programming Language :: Python :: 3.10", 30 | "Programming Language :: Python :: Implementation :: CPython", 31 | "Programming Language :: Python :: Implementation :: PyPy"]) 32 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/remykarem/pyterator/0235afc397a9fab0bdb6211a47c110087654e3d7/tests/__init__.py -------------------------------------------------------------------------------- /tests/test_pyterator.py: -------------------------------------------------------------------------------- 1 | from pyterator import iterate 2 | 3 | nums = [1, 2, 3, 4, 5] 4 | 5 | 6 | def test_iterate(): 7 | assert iterate(nums).to_list() == list(iter(nums)) -------------------------------------------------------------------------------- /tests/test_transforms.py: -------------------------------------------------------------------------------- 1 | from pyterator import iterate 2 | 3 | nums = [1, 2, 3, 4, 5] 4 | 5 | 6 | def test_map(): 7 | assert iterate(nums).map(lambda x: x+1).to_list() \ 8 | == list(map(lambda x: x+1, nums)) 9 | 10 | 11 | def test_map_shorthand_ops(): 12 | assert iterate(nums).map("+", 2).to_list() \ 13 | == list(map(lambda x: x+2, nums)) 14 | 15 | 16 | def test_map_custom_function(): 17 | def add(x, y): 18 | return x + 8*y 19 | 20 | assert iterate(nums).map(add, 5).to_list() \ 21 | == list(map(lambda x: add(x, 5), nums)) 22 | --------------------------------------------------------------------------------