README.md
[](http://rubygems.org/gems/slow_enumerator_tools)
[](http://rubygems.org/gems/slow_enumerator_tools)
[](https://travis-ci.org/ddfreyne/slow_enumerator_tools)
[](https://codeclimate.com/github/ddfreyne/slow_enumerator_tools)
[](https://codecov.io/gh/ddfreyne/slow_enumerator_tools)
# SlowEnumeratorTools
_SlowEnumeratorTools_ provides tools for transforming Ruby enumerators that produce data slowly and unpredictably (e.g. from a network source):
* `SlowEnumeratorTools.merge`: given a collection of enumerables, creates a new enumerator that yields elements from any of these enumerables as soon as they become available.
* `SlowEnumeratorTools.batch`: given an enumerable, creates a new enumerable that yields batches containing all elements currently available.
* `SlowEnumeratorTools.buffer`: given an enumerable and a number, will create a buffer of that number of elements and try to fill it up with as many elements from that enumerable, so that they can be yielded immediately.
## Installation
Add this line to your application's Gemfile:
```ruby
gem 'slow_enumerator_tools'
```
And then execute:
$ bundle
Or install it yourself as:
$ gem install slow_enumerator_tools
## Usage
### `SlowEnumeratorTools.merge`
Given a collection of enumerables, creates a new enumerator that yields elements from any of these enumerables as soon as they become available.
This is useful for combining multiple event streams into a single one.
```ruby
# Generate some slow enums
enums = []
enums << 5.times.lazy.map { |i| sleep(0.1 + rand * 0.2); [:a, i] }
enums << 5.times.lazy.map { |i| sleep(0.1 + rand * 0.2); [:b, i] }
enums << 5.times.lazy.map { |i| sleep(0.1 + rand * 0.2); [:c, i] }
# Merge and print
merged_enum = SlowEnumeratorTools.merge(enums)
merged_enum.each { |e| p e }
```
Example output:
```
[:b, 0]
[:a, 0]
[:b, 1]
[:c, 0]
[:a, 1]
[:b, 2]
[:c, 1]
[:c, 2]
[:a, 2]
[:b, 3]
[:c, 3]
[:b, 4]
[:a, 3]
[:c, 4]
[:a, 4]
```
### `SlowEnumeratorTools.batch`
Given an enumerable, creates a new enumerable that yields batches containing all elements currently available.
This is useful for fetching all outstanding events on an event stream, without blocking.
```ruby
# Generate a slow enum
enum = 4.times.lazy.map { |i| sleep(0.1); i }
# Batch
batch_enum = SlowEnumeratorTools.batch(enum)
# Wait until first batch is available
# … prints [0]
p batch_enum.next
# Give it enough time for the second batch to have accumulated more elements,
# … prints [1, 2]
sleep 0.25
p batch_enum.next
# Wait until final batch is available
# … prints [3]
p batch_enum.next
```
### `SlowEnumeratorTools.buffer`
Given an enumerable and a number, will create a buffer of that number of elements and try to fill it up with as many elements from that enumerable.
This is particularly useful when reading from a slow source and writing to a slow sink, because the two will be able to work concurrently.
```ruby
# Create (fake) articles enumerator
articles =
Enumerator.new do |y|
5.times do |i|
sleep 1
y << "Article #{i}"
end
end
# Buffer
articles = SlowEnumeratorTools.buffer(articles, 5)
# Print each article
# This takes 6 seconds, rather than 10!
articles.each do |a|
sleep 1
end
```
## Development
Install dependencies:
$ bundle
Run tests:
$ bundle exec rake
## Contributing
Bug reports and pull requests are welcome on GitHub at https://github.com/ddfreyne/slow_enumerator_tools. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
## License
The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
## Code of Conduct
Everyone interacting in the SlowEnumeratorTools project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/ddfreyne/slow_enumerator_tools/blob/master/CODE_OF_CONDUCT.md).