-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add position() #62
Comments
I would also like having position() on transcoding streams, so that I could use them with a parser that does something like this: len = read(io, UInt32)
offset = position(io)
while position(io) < offset + len
read_item(io)
end |
julia> using TranscodingStreams, Serialization
fid = TranscodingStream(Noop(), open("./__deleteme__.bin","w"))
serialize(fid, 1)
@show position(fid)
flush(fid)
@show position(fid)
# position(fid) = -9
# position(fid) = 9 |
Looking at CSV.jl again, it appears that implementing @bicycle1885 : I don't want to pressure you, but do you know when you will find time? With this information dependent packages could easier decide whether or not to create an intermediate work around. (e.g. load everything into a IOBuffer for RDatasets). |
It would be nice to have
Base.position(io::TranscodingStream)
to return the current position in the transcoded stream.There's already
state.total_out
, so it should not be impossible to keep track of the transcoded bytes since the last reset (+myabe adjust it w.r.tunread()
).The use case.
I came across this issue once again when trying to convert MLDatasets.jl to use CodecZlib.jl instead of the aging GZip.jl.
The gzipped files there are collections of images, each image occupying the same size. So, given image index, we can tell its exact position in the decoding stream.
When implementing
readimages(io, image_indices::AbstractVector)
, it was natural to definereadimage(io, image_index)
.To work,
readimage()
just needs to know the current stream position, so that it can calculate how many bytes to skip to get to the specified image.Without
position(io)
one would have to keep track of this information somewhere externally, which looks like an unnecessary complication to me.The text was updated successfully, but these errors were encountered: