Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add iterator interface to VideoReader objects #261

Merged
merged 4 commits into from
Aug 19, 2020

Conversation

galenlynch
Copy link
Collaborator

I have added an iterator interface to the VideoReader type, making collecting all frames of a video as simple as:

f = openvideo(fname)
imgs = collect(f)

The iterator returns a new array for each frame, as I think mutating the same array on subsequent calls to iterate would be surprising and lead to strange behavior when calling functions such as collect. Note that the VideoReader iterator is mutable, like Channel or Stateful iterators in Julia, and will resume iteration where it was last stopped, instead of at the beginning. I have added some text in the documentation explaining this behavior, and also added some tests.

I have added an iterator interface to the `VideoReader` type, making collecting
all frames of a video as simple as `f = openvideo(fname); imgs = collect(f)`.
The iterator returns a new array for each frame, as I think mutating the same
array on subsequent calls to `iterate` would be surprising and lead to strange
behavior when calling functions such as `collect`. Note that the VideoReader
iterator is mutable, like Channel or Stateful iterators in Julia, and will
resume iteration where it was last stopped, instead of at the beginning. I have
added some text in the documentation explaining this behavior, and also added
some tests.
@galenlynch
Copy link
Collaborator Author

I'm confused by these testing errors.

Copy link
Member

@IanButterworth IanButterworth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice addition. The test errors look to be out of memory related. The suggested change shouldn't be needed really, but may help on the more memory-limited CI machines?

test/avio.jl Outdated
Comment on lines 158 to 159
@test length(collect(v)) == VideoIO.TestVideos.videofiles[name].numframes
@test length(collect(v)) == 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
@test length(collect(v)) == VideoIO.TestVideos.videofiles[name].numframes
@test length(collect(v)) == 0
imgstack = collect(v)
@test length(imgstack) == VideoIO.TestVideos.videofiles[name].numframes
imgstack = nothing
@test length(collect(v)) == 0
GC.gc()

Copy link
Collaborator Author

@galenlynch galenlynch Aug 19, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah good points... if CI is memory limited I probably should not be using collect at all. I'll do some other test of the iterator interface that doesn't require storing all of the frames in memory at the same time.

Instead of collecting all frames into memory with `collect` to test the
`VideoReader` interface, I am instead testing the iterator interface with a
simple for loop. This should reduce the memory requirement for CI, and avoid
memory-related test errors on low memory CI VMs.
Instead of looping over the iterator to test that there are zero elements
remaining, it is simpler to just see if `iterate` returns `nothing`.
I have added a test to verify that frames returned by the `VideoReader`
iterator interface have distinct storage.
@codecov
Copy link

codecov bot commented Aug 19, 2020

Codecov Report

Merging #261 into master will decrease coverage by 3.19%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #261      +/-   ##
==========================================
- Coverage   77.13%   73.94%   -3.20%     
==========================================
  Files          14       14              
  Lines         608      614       +6     
==========================================
- Hits          469      454      -15     
- Misses        139      160      +21     
Impacted Files Coverage Δ
src/VideoIO.jl 52.27% <ø> (ø)
src/avio.jl 73.80% <100.00%> (-5.36%) ⬇️
src/util.jl 59.25% <0.00%> (-11.12%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0f211c2...0bea443. Read the comment docs.

@IanButterworth
Copy link
Member

Looks good now. Shall we merge?

@galenlynch
Copy link
Collaborator Author

Fine by me!

@IanButterworth IanButterworth merged commit 004431b into JuliaIO:master Aug 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants