csp - Golang style concurrency library based on Communicating Sequential Processes
105 |The csp package provides two concurrency primitives namely coroutines and channels which allow concurrent programming in the style of Golang.
167 |The concepts originate in Hoare's Communicating Sequential Processes while the syntax mimics the Golang implementation.
168 |The CSP concurrency model may be visualized as a set of independent processes (coroutines) sending and receiving messages to the named channels. The control flow in the coroutines is coordinated at the points of sending and receiving messages i.e. the coroutine may need to wait while trying to send or receive. 169 | Since it must work in a single-threaded interpreter, waiting is non-blocking. Instead of blocking a waiting coroutine gives way to other coroutines.
170 |This concurrency model may also be seen as a generalization of Unix named pipes where processes and pipes correspond to coroutines and channels.
171 |The unbuffered channel is a single value container that can be imagined as a rendez-vous venue where the sender must wait for the receiver to collect the message. 183 | By default channels are unbuffered.
The buffered channel behaves like a FIFO queue.
The buffered channel is ready for receive when it is not empty. 195 | The unbuffered channel is ready for receive if there exists a sender waiting with a message on this channel.
The buffered channel is ready for send when it is not full. 198 | The unbuffered channel is ready for send if there is no other sender already waiting on this channel. Note that
Channel is created with:
202 |::csp::channel chanVar ?size?
203 |Where the optional parameter size specifies the maximum number of messages that can be stored in the channel. When the channel is full the sender trying to send more messages to it must wait until any receiver offloads the channel. Waiting means that the sender gives way to other coroutines.
204 |If the size is zero (default) the created channel is unbuffered which means that the sender coroutine always waits for the receiver to collect the message.
205 |Channel may be closed with:
206 |channelObj close
207 |and is destroyed automatically when all messages are received (the channel is drained).
208 |Once the channel is created you can send to and receive from the channel.
When the channel is closed you can still receive from the channel but you cannot send to it. 217 | Trying to send to the closed channel throws an error. 218 | It is responsibility of the library user to close the unused channels.
The channel does not exist. 221 | After receiving all messages from the closed channel, the channel is destroyed. 222 | Trying to send to or receive from the destroyed channel throws an error.
Note that creating a large number of channels that are properly closed but not emptied may result in a memory leak.
Coroutine is a procedure executing concurrently with other coroutines. 228 | Coroutine may send messages to and receive messages from channels. Any coroutine may act as a sender or receiver at different times. If channel is not ready a coroutine waits by giving way to other coroutines. This makes the coroutine execution multiplexed at the points of sending to or receiving from channels.
229 |Coroutine is created with:
230 |::csp::go procName ?args?
231 |where procName is the name of the existing Tcl procedure that will be run as a coroutine. 232 | You can create many coroutines from a single Tcl procedure, possibly called with different arguments.
233 |Coroutine is destroyed when its execution ends.
234 |We reuse the term coroutine known from Tcl (modelled on Lua) coroutines, but they are are not equivalent. csp coroutines are implemented in terms of Tcl coroutines and it's better not to mix csp and Tcl coroutines in a single program.
Create a coroutine by calling procName with arguments args. Returns internal name of the coroutine.
Create a channel object that will be further referred as channelObj. The name of the object is contained in variable channelVar.
243 |Variable channelVar that will be created and will contiain the channel object name.
Size of the channel buffer understood as the maximum number of messages that can be buffered in the channel. If size is zero (default) the channel is unbuffered.
Returns channel object name.
Close the channel channelObj. Returns empty string.
Send msg to channel channelObj in a coroutine. Returns empty string.
Send msg to channel channelObj in a script (in the Tcl program main control flow). It is implemented using vwait and has many limitations. Use with care and only in simple scenarios. Returns empty string.
Receive from channel channelObj in a coroutine. Returns the message received from the channel.
Receive from channel channelObj in a script (in the Tcl program main control flow). Returns the message received from the channel.
Evaluate set of channels to find which channels are ready and run corresponding block of code. Returns the result of evaluation of the block of code.
263 |Operation takes one of 3 forms:
267 |<- channelObj
268 |for evaluating whether the channelObj is ready for receive, or
269 |channelObj <-
270 |for evaluating whether the channelObj is ready for send, or
271 |default
272 |for evaluating default case if no channel is ready.
Block of code to be evaluated.
The select command provides a way to handle multiple channels. It is a switch like statement where channels are evaluated for readiness. The select command makes the coroutine wait until at least one channel is ready. If multiple channels can proceed, select chooses pseudo-randomly. A default clause, if present, executes immediately if no channel is ready.
Receive from channel until closed in a coroutine.
279 |This is a foreach like construct that iterates by receiving messages from channel one by one until channel is closed. If channel is not ready for receive, range waits.
Receive from channel until closed in the main control flow.
282 |A version of range command that can be used outside of a coroutine. It is implemented using vwait and has many limitations. Use with care and only in simple scenarios.
Create a receive-only channel with scheduled message in interval milliseconds. Trying to receive from the channel will cause the coroutine to wait interval milliseconds since creation. Eventually the received message is a Unix epoch time in microseconds. After receiving the message the channel is closed and destroyed.
285 |Returns the created channel.
Create a receive-only channel with scheduled messages every interval milliseconds.
288 |Returns the created channel. 289 | The optional closeafter argument determines when the channel is closed. It may take one of the 2 forms:
290 |integerNumber that specifies the number of milliseconds after which the channel will be closed
#integerNumber that specifies number of messages after which the channel will be closed
If closeafter argument is not provided, the ticker channel emits messages endlessly.
Creates a channel and returns a new coroutine that may be called with a single argument. The coroutine is meant for integration with callback-driven code and to be used in place of one-time callback. The channel is placed in channelVar and will be destroyed after receiving a single message. The single argument passed to the callback will be available to receive from the created channel.
298 |Note that there is a limitation in replacing callbacks with -> command: only a single- or zero- argument callbacks can be replaced. In case of zero-argument callbacks an empty string is sent to the channel.
Creates a buffered channel of size size and returns a new coroutine that may be used in place of a callback. The coroutine may be called many times and the callback arguments are internally sent to the created channel.
301 |Note that there is a limitation in replacing callbacks with -> command: only a single- or zero- argument callbacks can be replaced. In case of zero-argument callbacks an empty string is sent to the channel.
Receive messages from fromChannel and send them to toChannel.
Simple message passing over an unbuffered channel
309 |
310 | package require csp
311 | namespace import csp::*
312 |
313 | proc sender1 {ch} {
314 | foreach i {1 2 3 4} {
315 | puts "Sending $i"
316 | $ch <- $i
317 | }
318 | puts "Closing channel"
319 | $ch close
320 | }
321 |
322 | proc receiver1 {ch} {
323 | while 1 {
324 | puts "Receiving [<- $ch]"
325 | }
326 | }
327 |
328 | # create unbuffered (rendez-vous) channel
329 | channel ch
330 | go sender1 $ch
331 | go receiver1 $ch
332 |
333 | vwait forever
334 |
335 | Output:
336 |337 | Sending 1 338 | Receiving 1 339 | Sending 2 340 | Receiving 2 341 | Sending 3 342 | Receiving 3 343 | Sending 4 344 | Receiving 4 345 | Closing channel 346 |347 |
The communication between the coroutines is coordinated because the channel is unbuffered. 348 | The sender waits for the receiver.
349 |Simple message passing over a buffered channel
352 |
353 | package require csp
354 | namespace import csp::*
355 |
356 | proc sender1 {ch} {
357 | foreach i {1 2 3 4} {
358 | puts "Sending $i"
359 | $ch <- $i
360 | }
361 | puts "Closing channel"
362 | $ch close
363 | }
364 |
365 | proc receiver1 {ch} {
366 | while 1 {
367 | puts "Receiving [<- $ch]"
368 | }
369 | }
370 |
371 | # create buffered channel of size 2
372 | channel ch 2
373 | go sender1 $ch
374 | go receiver1 $ch
375 |
376 | vwait forever
377 |
378 | Output:
379 |380 | Sending 1 381 | Sending 2 382 | Sending 3 383 | Receiving 1 384 | Receiving 2 385 | Sending 4 386 | Closing channel 387 | Receiving 3 388 | Receiving 4 389 | Error: Cannot receive from a drained (empty and closed) channel ::csp::Channel#1 390 |391 |
Since the channel is buffered of size 2, the sender waits only on the third attempt.
392 |Note that the channel was closed but we still receive messages. Only after the channel was emptied, trying to receive from the channel throws an error.
393 |Using range for receiving from the channel until closed.
396 |We can prevent throwing the error in the previous example by using the range command instead of iterating blindly with while. 397 | Also if the channel is buffered we can send all messages first and iterate to receive using range in a single coroutine.
398 |
399 | package require csp
400 | namespace import csp::*
401 |
402 | proc senderreceiver {ch} {
403 | foreach i {1 2 3 4} {
404 | puts "Sending $i"
405 | $ch <- $i
406 | }
407 | puts "Closing channel"
408 | $ch close
409 | range msg $ch {
410 | puts "Message $msg"
411 | }
412 | puts "Received all"
413 | }
414 |
415 | channel ch 10
416 | go senderreceiver $ch
417 |
418 | vwait forever
419 |
420 | Output:
421 |422 | Sending 1 423 | Sending 2 424 | Sending 3 425 | Sending 4 426 | Closing channel 427 | Message 1 428 | Message 2 429 | Message 3 430 | Message 4 431 | Received all 432 |433 |
Channels can be used to coordinate future events. We use after to create coroutine that will send to the channel.
436 |Instead of using direct callback which cannot keep local state we consume events in adder which can keep sum in local variable.
437 |
438 | package require csp
439 | namespace import csp::*
440 |
441 | proc adder {ch} {
442 | set sum 0
443 | while 1 {
444 | set number [<- $ch]
445 | incr sum $number
446 | puts "adder received $number. The sum is $sum"
447 | }
448 | }
449 |
450 | proc trigger {ch number} {
451 | $ch <- $number
452 | }
453 |
454 | channel ch
455 | go adder $ch
456 | after 1000 go trigger $ch 1
457 | after 3000 go trigger $ch 3
458 | after 5000 go trigger $ch 5
459 | puts "Enter event loop"
460 |
461 | vwait forever
462 |
463 | Output:
464 |465 | Enter event loop 466 | adder received 1. The sum is 1 467 | adder received 3. The sum is 4 468 | adder received 5. The sum is 9 469 |470 |
Use timer to create a channel supplying scheduled messages in the future.
473 |
474 | package require csp
475 | namespace import csp::*
476 |
477 | proc future {ch} {
478 | try {
479 | puts "future happened at [<- $ch]"
480 | puts "try to receive again:"
481 | puts "[<- $ch]"
482 | } on error {out err} {
483 | puts "error: $out"
484 | }
485 | }
486 |
487 | timer ch 2000
488 | go future $ch
489 | puts "Enter event loop at [clock microseconds]"
490 |
491 | vwait forever
492 |
493 | Output:
494 |495 | Enter event loop at 1434472163190638 496 | future happened at 1434472165189759 497 | try to receive again: 498 | error: Cannot receive from a drained (empty and closed) channel ::csp::Channel#1 499 |500 |
Instead of scheduling events with after we can use timer to create a special receive only channel. There will be only one message send to this channel after the specified time so we can pass this channel to another coroutine that will wait for that message. The message from the timer channel represents unix epoch time in microseconds. The timer channel will be automatically destroyed after first receive so trying to receive again will throw an error.
501 |Using ticker we can create receive only channel from which we can consume timestamp messages at regular intervals.
504 |
505 | package require csp
506 | namespace import csp::*
507 |
508 | proc future {ch} {
509 | set count 0
510 | while 1 {
511 | incr count
512 | puts "future $count received at [<- $ch]"
513 | }
514 | }
515 |
516 | ticker ch 1000
517 | go future $ch
518 | puts "Enter event loop at [clock microseconds]"
519 |
520 | vwait forever
521 |
522 | Output:
523 |524 | Enter event loop at 1434472822879684 525 | future 1 received at 1434472823879110 526 | future 2 received at 1434472824882163 527 | future 3 received at 1434472825884246 528 | ... 529 |530 |
ticker command returns the created channel so we can use it in place in combination with range to further simplify the example
533 |
534 | package require csp
535 | namespace import csp::*
536 |
537 | proc counter {} {
538 | range t [ticker ch 1000] {
539 | puts "received $t"
540 | }
541 | }
542 |
543 | go counter
544 |
545 | vwait forever
546 |
547 | Output:
548 |549 | received 1434474325947677 550 | received 1434474326950822 551 | received 1434474327952904 552 | ... 553 |554 |
Another example of using ticker to implement the canonical countdown counter from Tcl wiki.
557 |
558 | package require Tk
559 | package require csp
560 | namespace import csp::*
561 |
562 | proc countdown {varName} {
563 | upvar $varName var
564 | range _ [ticker t 1000 #10] {
565 | incr var -1
566 | }
567 | }
568 |
569 | set count 10
570 | label .counter -font {Helvetica 72} -width 3 -textvariable count
571 | grid .counter -padx 100 -pady 100
572 | go countdown count
573 |
574 | Closing the channel by another scheduled event breaks the range loop
577 |
578 | package require csp
579 | namespace import csp::*
580 |
581 | proc counter {ch} {
582 | range t $ch {
583 | puts "received $t"
584 | }
585 | puts "counter exit"
586 | }
587 |
588 | ticker ch 1000
589 | go counter $ch
590 | after 4500 $ch close
591 | puts "Enter event loop at [clock microseconds]"
592 |
593 | vwait forever
594 |
595 | Output:
596 |597 | Enter event loop at 1434474384645704 598 | received 1434474385644900 599 | received 1434474386648105 600 | received 1434474387650088 601 | received 1434474388652345 602 | counter exit 603 |604 |
Redirect callback call argument to a channel using -> command.
607 |
608 | package require http
609 | package require csp
610 | namespace import csp::*
611 |
612 | proc main {} {
613 | http::geturl http://securitykiss.com/rest/now -command [-> ch]
614 | puts "fetched: [http::data [<- $ch]]"
615 | }
616 |
617 | go main
618 |
619 | vwait forever
620 |
621 | Output:
622 |623 | fetched: 1434474568 624 |625 |
csp package makes it easy to integrate channels and coroutines with existing event driven code. 626 | Using the -> utility command we can make channels work with callback driven commands and at the same time avoid callback hell.
627 |-> ch creates a channel ch and returns a new coroutine that may be used in place of a callback. 628 | The channel will be destroyed after receiving a single value. 629 | The single argument passed to the callback will be available to receive from the created channel.
630 |Such code organization promotes local reasoning - it helps writing linear code with local state kept in proc variables. Otherwise the callback would require keeping state in global variables.
631 |Note that there is a limitation in replacing callbacks with -> command: only a single- or zero- argument callbacks can be replaced. 632 | In case of zero-argument callbacks an empty string is sent to the channel.
633 |Note that there is no symmetry in <- <-! -> ->> commands. Every one of them has a different purpose.
634 |Use select command to choose ready channels.
637 |
638 | package require http
639 | package require csp
640 | namespace import csp::*
641 |
642 | proc main {} {
643 | http::geturl http://securitykiss.com/rest/slow/now -command [-> ch1]
644 | http::geturl http://securitykiss.com/rest/slow/now -command [-> ch2]
645 | select {
646 | <- $ch1 {
647 | puts "from first request: [http::data [<- $ch1]]"
648 | }
649 | <- $ch2 {
650 | puts "from second request: [http::data [<- $ch2]]"
651 | }
652 | }
653 | }
654 |
655 | go main
656 |
657 | vwait forever
658 |
659 | Output:
660 |661 | from first request: 1434483100 662 |663 |
Previous example with callback channels does not extend to making parallel http requests because one waiting channel would prevent receiving from the other. 664 | The select command chooses which of a set of possible send or receive operations will proceed. In this example select command examines two callback channels and depending on which one is ready for receive first, it evaluates corresponding body block.
665 |Combine timer created channel with select to enforce timeouts.
668 |
669 | package require http
670 | package require csp
671 | namespace import csp::*
672 |
673 | proc main {} {
674 | http::geturl http://securitykiss.com/rest/slow/now -command [-> ch1]
675 | http::geturl http://securitykiss.com/rest/slow/now -command [-> ch2]
676 | timer t1 400
677 | select {
678 | <- $ch1 {
679 | puts "from first request: [http::data [<- $ch1]]"
680 | }
681 | <- $ch2 {
682 | puts "from second request: [http::data [<- $ch2]]"
683 | }
684 | <- $t1 {
685 | puts "requests timed out at [<- $t1]"
686 | }
687 | }
688 | }
689 |
690 | go main
691 |
692 | vwait forever
693 |
694 | Output:
695 |696 | requests timed out at 1434484003634953 697 |698 |
Since select chooses from the set of channels whichever is ready first, by adding the timer created channel to select from, we can implement timeout as in the example above.
699 |Use select with the default clause.
702 |
703 | package require http
704 | package require csp
705 | namespace import csp::*
706 |
707 | proc DisplayResult {ch1 ch2} {
708 | set result [select {
709 | <- $ch1 {
710 | http::data [<- $ch1]
711 | }
712 | <- $ch2 {
713 | http::data [<- $ch2]
714 | }
715 | default {
716 | subst "no response was ready"
717 | }
718 | }]
719 | puts "DisplayResult: $result"
720 | }
721 |
722 | proc main {} {
723 | http::geturl http://securitykiss.com/rest/slow/now -command [-> ch1]
724 | http::geturl http://securitykiss.com/rest/slow/now -command [-> ch2]
725 | after 400 go DisplayResult $ch1 $ch2
726 | }
727 |
728 | go main
729 |
730 | vwait forever
731 |
732 | Output:
733 |734 | DisplayResult: no response was ready 735 |736 |
select command is potentially waiting if no channel is ready. Sometimes we need to proceed no matter what so select makes it possible to return without waiting if the default clause is provided. This example also shows that select has a return value. In this case the result returned by select is either HTTP response or the value specified in the default block if no channel is ready.
737 |Funnel multiple channels into a single channel using forward command.
740 |
741 | package require http
742 | package require csp
743 | namespace import csp::*
744 |
745 | proc main {} {
746 | set urls {
747 | http://securitykiss.com
748 | http://meetup.com
749 | http://reddit.com
750 | http://google.com
751 | http://twitter.com
752 | http://bitcoin.org
753 | }
754 | channel f
755 | foreach url $urls {
756 | http::geturl $url -method HEAD -command [-> ch]
757 | forward $ch $f
758 | }
759 | after 200 $f close
760 | range token $f {
761 | upvar #0 $token state
762 | puts "$state(http)\t$state(url)"
763 | }
764 | puts "main exit"
765 | }
766 |
767 | go main
768 |
769 | vwait forever
770 |
771 | Output:
772 |773 | HTTP/1.1 302 Found http://google.com/ 774 | HTTP/1.1 301 Moved Permanently http://reddit.com/ 775 | HTTP/1.1 301 Moved Permanently http://securitykiss.com/ 776 | main exit 777 |778 |
When we want to listen on many channels, especially when they are dynamically created for example per URL as in the above example, select command becomes awkward because it requires specifying logic for every channel.
779 |In the example above we spawn a HTTP request for every URL and forward messages from individual "callback channels" into the single "funnel channel" f. In this way the responses are available in a single channel so we can apply common logic to the results. We also set the timeout for the requests by closing the "funnel channel" after some time. Responses that don't make it within a specified timeout are ignored.
780 |Redirect callback multi call argument to a long-lived channel using ->> command.
783 |
784 | package require Tk
785 | package require csp
786 | namespace import csp::*
787 |
788 | proc main {} {
789 | set number 5
790 | frame .f
791 | button .f.left -text <<< -command [->> chleft]
792 | label .f.lbl -font {Helvetica 24} -text $number
793 | button .f.right -text >>> -command [->> chright]
794 | grid .f.left .f.lbl .f.right
795 | grid .f
796 | while 1 {
797 | select {
798 | <- $chleft {
799 | <- $chleft
800 | incr number -1
801 | }
802 | <- $chright {
803 | <- $chright
804 | incr number
805 | }
806 | }
807 | .f.lbl configure -text $number
808 | }
809 | }
810 |
811 | go main
812 |
813 | In previous examples the -> command created short-lived disposable callback channels that could be received from only once. 814 | Often an existing command require a callback that will be called many times over long period of time. In such case ->> comes to play. 815 | It returns a coroutine that may be called many times in place of the callback. Callback argument is passed to the newly created buffered channel that can be later received from to consume the messages (callback arguments).
816 |In this example similar functionality could be achieved in a simpler way using -textvariable on label but it would require a global variable instead of local number.
817 |The same limitations regarding callback arguments arity apply as for the -> command.
818 |Note that there is no symmetry in <- <-! -> ->> commands. Every one of them has a different purpose.
819 |Channel operations like <- and range can be used only in coroutines. Using coroutines for channel driven coordination is the recommended way of using csp package.
822 |It may happen that we need to use channels outside of coroutines. It is possible with corresponding <-! and range! commands but there are caveats. 823 | The "bang" terminated commands are implemented using vwait nested calls and have many limitations. Thus they should be used with extra care and only in simple scenarios. Especially it is not guaranteed that they will work correctly if used inside callbacks.
824 |In this example we show a simple scenario where receiving from the channel in the main script control flow makes sense as a way to synchronize coroutine termination.
825 |
826 | package require http
827 | package require csp
828 | namespace import csp::*
829 |
830 | proc worker {ch_quit} {
831 | http::geturl http://securitykiss.com/rest/now -command [-> ch]
832 | puts "fetched: [http::data [<- $ch]]"
833 | $ch_quit <- 1
834 | }
835 |
836 | # termination coordination channel
837 | channel ch_quit
838 |
839 | go worker $ch_quit
840 |
841 | <-! $ch_quit
842 |
843 | Output:
844 |845 | fetched: 1434556219 846 |847 |
Without the last line the script would exit immediately without giving the coroutine a chance to fetch the url.
848 |Following the "bang" terminated command trail, this example shows how range! command may further simplify the previous countdown counter example.
851 |
852 | package require Tk
853 | package require csp
854 | namespace import csp::*
855 |
856 | set count 5
857 | label .counter -font {Helvetica 72} -width 3 -textvariable count
858 | grid .counter -padx 100 -pady 100
859 | range! _ [ticker t 1000 #$count] {
860 | incr count -1
861 | }
862 |
863 | A more complex example using the already discussed constructs.
866 |
867 | # A simple web crawler/scraper demonstrating the csp style programming in Tcl
868 | # In this example we have 2 coroutines: a crawler and a parser communicating over 2 channels.
869 | # The crawler receives the url to process from the urls channel and spawns a http request
870 | # Immediately sends the pair: (url, callback channel from http request)
871 | # into a pending requests channel for further processing by the parser.
872 | # The parser receives the http token from the received callback channel
873 | # and fetches the page content from the url in order to extract more urls.
874 | # The new urls are sent to the urls channel where the crawler takes over again.
875 |
876 | package require http
877 | package require csp
878 | namespace import csp::*
879 |
880 | # The crawler coroutine is initialized with 3 channels:
881 | # urls - channel with urls waiting to process
882 | # requests - channel with pending http requests
883 | # quit - synchronization channel to communicate end of coroutine
884 | proc crawler {urls requests quit} {
885 | # list of visited urls
886 | set visited {}
887 | range url $urls {
888 | if {$url ni $visited} {
889 | http::geturl $url -command [-> req]
890 | lappend visited $url
891 | # note we are passing channel object over a channel
892 | $requests <- [list $url $req]
893 | }
894 | }
895 | $quit <- 1
896 | }
897 |
898 |
899 | # The parser coroutine is initialized with 3 channels:
900 | # urls - channel with urls waiting to process
901 | # requests - channel with pending http requests
902 | # quit - synchronization channel to communicate end of coroutine
903 | proc parser {urls requests quit} {
904 | set count 0
905 | range msg $requests {
906 | lassign $msg url req
907 | timer timeout 5000
908 | select {
909 | <- $req {
910 | set token [<- $req]
911 | set data [http::data $token]
912 | puts "Fetched URL $url with content size [string length $data] bytes"
913 | foreach {_ href} [regexp -nocase -all -inline {href="(.*?)"} $data] {
914 | if {![string match http:* $href] && ![string match mailto:* $href]} {
915 | # catch error if channel has been closed
916 | $urls <- [create_url $url $href]
917 | }
918 | }
919 | }
920 | <- $timeout {
921 | <- $timeout
922 | puts "Request to $url timed out"
923 | }
924 | }
925 | # we stop after fetching 10 urls
926 | if {[incr count] >= 10} {
927 | $urls close
928 | $requests close
929 | }
930 | }
931 | $quit <- 1
932 | }
933 |
934 | # utility function to construct urls
935 | proc create_url {current href} {
936 | regexp {(http://[^/]*)(.*)} $current _ base path
937 | if {[string match /* $href]} {
938 | return $base$href
939 | } else {
940 | return $current$href
941 | }
942 | }
943 |
944 |
945 | # channel containing urls to process
946 | # this channel must have rather large buffer so that the urls to crawl can queue
947 | channel urls 10000
948 | # channel containing (url req) pairs representing pending http requests
949 | # size of this channel determines parallelism i.e. the maximum number of pending requests at the same time
950 | channel requests 3
951 | # coordination channels that make the main program wait until coroutines end
952 | channel crawler_quit
953 | channel parser_quit
954 | go crawler $urls $requests $crawler_quit
955 | go parser $urls $requests $parser_quit
956 |
957 | # send the seed url to initiate crawling
958 | $urls <-! "http://www.tcl.tk/man/tcl8.6/"
959 |
960 | # Gracefully exit - wait for coroutines to complete
961 | <-! $crawler_quit
962 | <-! $parser_quit
963 |
964 | In particular it is worth noting:
965 |it is possible to pass a channel object over another channel
use of quit synchronization channel to communicate end of coroutine
closing channels as a way to terminate range iteration
actors, callback, channel, concurrency, csp, golang
974 |Concurrency
977 |Copyright © 2015 SecurityKISS Ltd <open.source@securitykiss.com> - MIT License - Feedback and bug reports are welcome
980 |