Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add keyword arguments to IOBuffer's constructors #25872

Merged
merged 5 commits into from
Feb 6, 2018
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions base/deprecated.jl
Original file line number Diff line number Diff line change
Expand Up @@ -1415,6 +1415,11 @@ end

@deprecate which(s::Symbol) which(Main, s)

@deprecate IOBuffer(data::AbstractVector{UInt8}, read::Bool, write::Bool=false, maxsize::Integer=typemax(Int)) IOBuffer(data, read=read, write=write, maxsize=maxsize)
@deprecate IOBuffer(read::Bool, write::Bool) IOBuffer(read=read, write=write)
@deprecate IOBuffer(maxsize::Integer) IOBuffer(read=true, write=true, maxsize=maxsize)


# END 0.7 deprecations

# BEGIN 1.0 deprecations
Expand Down
2 changes: 1 addition & 1 deletion base/int.jl
Original file line number Diff line number Diff line change
Expand Up @@ -539,7 +539,7 @@ end
macro big_str(s)
if '_' in s
# remove _ in s[2:end-1]
bf = IOBuffer(lastindex(s))
bf = IOBuffer(maxsize=lastindex(s))
print(bf, s[1])
for c in SubString(s, 2, lastindex(s)-1)
c != '_' && print(bf, c)
Expand Down
110 changes: 55 additions & 55 deletions base/iobuffer.jl
Original file line number Diff line number Diff line change
Expand Up @@ -32,15 +32,27 @@ StringVector(n::Integer) = unsafe_wrap(Vector{UInt8}, _string_n(n))
# IOBuffers behave like Files. They are typically readable and writable. They are seekable. (They can be appendable).

"""
IOBuffer([data, ][readable::Bool=true, writable::Bool=false[, maxsize::Int=typemax(Int)]])
IOBuffer([data::AbstractVector{UInt8}]; keywords...) -> IOBuffer

Create an `IOBuffer`, which may optionally operate on a pre-existing array. If the
readable/writable arguments are given, they restrict whether or not the buffer may be read
from or written to respectively. The last argument optionally specifies a size beyond which
the buffer may not be grown.
Create an in-memory I/O stream, which may optionally operate on a pre-existing array.

It may take optional keyword arguments:
- `read`, `write`, `append`: restricts operations to the buffer; see `open` for details.
- `truncate`: truncates the buffer size to zero length.
- `maxsize`: specifies a size beyond which the buffer may not be grown.

When `data` is given, the buffer will be both readable and writable by default.
Copy link
Member

@JeffBezanson JeffBezanson Feb 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is true --- the keyword arguments default to nothing, which open_flags interprets as read-only. And that's probably a good thing; for example we have

IOBuffer(str::String) = IOBuffer(unsafe_wrap(Vector{UInt8}, str))

which we want to remain read-only.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right. I meant "When data is not given, ...". Fixed.


# Examples
```jldoctest
julia> io = IOBuffer();

julia> write(io, "JuliaLang is a GitHub organization.", " It has many members.")
56

julia> String(take!(io))
"JuliaLang is a GitHub organization. It has many members."

julia> io = IOBuffer("JuliaLang is a GitHub organization.")
IOBuffer(data=UInt8[...], readable=true, writable=false, seekable=true, append=false, size=35, maxsize=Inf, ptr=1, mark=-1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The examples don't include the truncate flag. As an aside, I guess I should write the code to figure out a minimal set of keywords to reproduce a particular set of open flags – this output is a bit on the long side.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. I added some examples in #25919.


Expand All @@ -50,7 +62,7 @@ julia> read(io, String)
julia> write(io, "This isn't writable.")
ERROR: ArgumentError: ensureroom failed, IOBuffer is not writeable

julia> io = IOBuffer(UInt8[], true, true, 34)
julia> io = IOBuffer(UInt8[], read=true, write=true, maxsize=34)
IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=0, maxsize=34, ptr=1, mark=-1)

julia> write(io, "JuliaLang is a GitHub organization.")
Expand All @@ -60,57 +72,45 @@ julia> String(take!(io))
"JuliaLang is a GitHub organization"
```
"""
IOBuffer(data::AbstractVector{UInt8}, readable::Bool=true, writable::Bool=false, maxsize::Integer=typemax(Int)) =
GenericIOBuffer(data, readable, writable, true, false, maxsize)
function IOBuffer(readable::Bool, writable::Bool)
b = IOBuffer(StringVector(32), readable, writable)
b.data[:] = 0
b.size = 0
return b
function IOBuffer(
data::AbstractVector{UInt8};
read::Union{Bool,Nothing}=nothing,
write::Union{Bool,Nothing}=nothing,
append::Union{Bool,Nothing}=nothing,
truncate::Union{Bool,Nothing}=nothing,
maxsize::Integer=typemax(Int))
if maxsize < 0
throw(ArgumentError("negative maxsize: $(maxsize)"))
end
flags = open_flags(read=read, write=write, append=append, truncate=truncate)
buf = GenericIOBuffer(data, flags.read, flags.write, true, flags.append, Int(maxsize))
if flags.truncate
buf.size = 0
end
return buf
end

"""
IOBuffer() -> IOBuffer

Create an in-memory I/O stream, which is both readable and writable.

# Examples
```jldoctest
julia> io = IOBuffer();

julia> write(io, "JuliaLang is a GitHub organization.", " It has many members.")
56

julia> String(take!(io))
"JuliaLang is a GitHub organization. It has many members."
```
"""
IOBuffer() = IOBuffer(true, true)

"""
IOBuffer(size::Integer)

Create a fixed size IOBuffer. The buffer will not grow dynamically.

# Examples
```jldoctest
julia> io = IOBuffer(12)
IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=0, maxsize=12, ptr=1, mark=-1)

julia> write(io, "Hello world.")
12

julia> String(take!(io))
"Hello world."

julia> write(io, "Hello world again.")
12

julia> String(take!(io))
"Hello world "
```
"""
IOBuffer(maxsize::Integer) = (x=IOBuffer(StringVector(maxsize), true, true, maxsize); x.size=0; x)
function IOBuffer(;
read::Union{Bool,Nothing}=true,
write::Union{Bool,Nothing}=true,
append::Union{Bool,Nothing}=nothing,
truncate::Union{Bool,Nothing}=true,
maxsize::Integer=typemax(Int))
size = maxsize == typemax(Int) ? 32 : Int(maxsize)
flags = open_flags(read=read, write=write, append=append, truncate=truncate)
buf = IOBuffer(
StringVector(size),
read=flags.read,
write=flags.write,
append=flags.append,
truncate=flags.truncate,
maxsize=maxsize)
buf.data[:] = 0
if flags.truncate
buf.size = 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not important, just wondering: since truncate is also passed to the constructor above, is this redundant?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is. Fixed in #25919.

end
return buf
end

# PipeBuffers behave like Unix Pipes. They are typically readable and writable, they act appendable, and are not seekable.

Expand Down
2 changes: 1 addition & 1 deletion base/printf.jl
Original file line number Diff line number Diff line change
Expand Up @@ -1114,7 +1114,7 @@ function bigfloat_printf(out, d::BigFloat, flags::String, width::Int, precision:
if precision >= 0
fmt_len += ndigits(precision)+1
end
fmt = IOBuffer(fmt_len)
fmt = IOBuffer(maxsize=fmt_len)
write(fmt, '%')
write(fmt, flags)
if width > 0
Expand Down
4 changes: 2 additions & 2 deletions base/strings/basic.jl
Original file line number Diff line number Diff line change
Expand Up @@ -512,7 +512,7 @@ isascii(s::AbstractString) = all(isascii, s)
## string map, filter, has ##

function map(f, s::AbstractString)
out = IOBuffer(StringVector(sizeof(s)), true, true)
out = IOBuffer(StringVector(sizeof(s)), read=true, write=true)
Copy link
Member

@stevengj stevengj Feb 7, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passing a StringVector like this seems to be pretty common. It would be good to add a sizehint keyword argument so that you could just do IOBuffer(sizehint=sizeof(s)) here and in similar places.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I'll take care of it.

truncate(out, 0)
for c in s
c′ = f(c)
Expand All @@ -525,7 +525,7 @@ function map(f, s::AbstractString)
end

function filter(f, s::AbstractString)
out = IOBuffer(StringVector(sizeof(s)), true, true)
out = IOBuffer(StringVector(sizeof(s)), read=true, write=true)
truncate(out, 0)
for c in s
f(c) && write(out, c)
Expand Down
6 changes: 3 additions & 3 deletions base/strings/io.jl
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ julia> sprint(showcompact, 66.66666)
```
"""
function sprint(f::Function, args...; context=nothing, sizehint::Integer=0)
s = IOBuffer(StringVector(sizehint), true, true)
s = IOBuffer(StringVector(sizehint), read=true, write=true)
# specialized version of truncate(s,0)
s.size = 0
s.ptr = 1
Expand All @@ -100,7 +100,7 @@ tostr_sizehint(x::Float32) = 12

function print_to_string(xs...; env=nothing)
# specialized for performance reasons
s = IOBuffer(StringVector(tostr_sizehint(xs[1])), true, true)
s = IOBuffer(StringVector(tostr_sizehint(xs[1])), read=true, write=true)
# specialized version of truncate(s,0)
s.size = 0
s.ptr = 1
Expand Down Expand Up @@ -436,7 +436,7 @@ Returns:
function unindent(str::AbstractString, indent::Int; tabwidth=8)
indent == 0 && return str
# Note: this loses the type of the original string
buf = IOBuffer(StringVector(sizeof(str)), true, true)
buf = IOBuffer(StringVector(sizeof(str)), read=true, write=true)
truncate(buf,0)
cutting = true
col = 0 # current column (0 based)
Expand Down
2 changes: 1 addition & 1 deletion base/strings/util.jl
Original file line number Diff line number Diff line change
Expand Up @@ -383,7 +383,7 @@ function replace(str::String, pat_repl::Pair; count::Integer=typemax(Int))
i = a = firstindex(str)
r = coalesce(findnext(pattern,str,i), 0)
j, k = first(r), last(r)
out = IOBuffer(StringVector(floor(Int, 1.2sizeof(str))), true, true)
out = IOBuffer(StringVector(floor(Int, 1.2sizeof(str))), read=true, write=true)
out.size = 0
out.ptr = 1
while j != 0
Expand Down
2 changes: 1 addition & 1 deletion stdlib/Dates/src/io.jl
Original file line number Diff line number Diff line change
Expand Up @@ -481,7 +481,7 @@ end

function format(dt::TimeType, fmt::DateFormat, bufsize=12)
# preallocate to reduce resizing
io = IOBuffer(Vector{UInt8}(uninitialized, bufsize), true, true)
io = IOBuffer(Vector{UInt8}(uninitialized, bufsize), read=true, write=true)
format(io, dt, fmt)
String(io.data[1:io.ptr - 1])
end
Expand Down
4 changes: 2 additions & 2 deletions test/functional.jl
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ end
@test isequal(map(sqrt, 2:6), [sqrt(i) for i in 2:6])

# map on ranges should evaluate first value only once (#4453)
let io=IOBuffer(3)
let io=IOBuffer(maxsize=3)
map(x->print(io,x), 1:2)
@test String(take!(io))=="12"
end
Expand Down Expand Up @@ -176,4 +176,4 @@ end
@test res isa Vector{Union{Bool, T}}
res = map(f, y)
@test res isa Vector{Union{Bool, T}}
end
end
10 changes: 5 additions & 5 deletions test/iobuffer.jl
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,7 @@ end

@testset "pr #11554" begin
io = IOBuffer(SubString("***αhelloworldω***", 4, 16))
io2 = IOBuffer(Vector{UInt8}(b"goodnightmoon"), true, true)
io2 = IOBuffer(Vector{UInt8}(b"goodnightmoon"), read=true, write=true)

@test read(io, Char) == 'α'
@test_throws ArgumentError write(io,"!")
Expand Down Expand Up @@ -227,7 +227,7 @@ end

# issue #11917
# (previous tests triggered this sometimes, but this should trigger nearly all the time)
let io = IOBuffer(0)
let io = IOBuffer(maxsize=0)
write(io, fill(0x01, 1048577))
end

Expand Down Expand Up @@ -289,11 +289,11 @@ end
end

@testset "Test constructor with a generic type argument." begin
io = IOBuffer(Int16(10))
io = IOBuffer(maxsize=Int16(10))
@test io isa IOBuffer
io = IOBuffer(Int32(10))
io = IOBuffer(maxsize=Int32(10))
@test io isa IOBuffer
io = IOBuffer(Int64(10))
io = IOBuffer(maxsize=Int64(10))
@test io isa IOBuffer
end

Expand Down
2 changes: 1 addition & 1 deletion test/read.jl
Original file line number Diff line number Diff line change
Expand Up @@ -343,7 +343,7 @@ for (name, f) in l
@test read("$filename.to", String) == text

verbose && println("$name write(::IOBuffer, ...)")
to = IOBuffer(Vector{UInt8}(codeunits(text)), false, true)
to = IOBuffer(Vector{UInt8}(codeunits(text)), read=false, write=true)
write(to, io())
@test String(take!(to)) == text

Expand Down
2 changes: 1 addition & 1 deletion test/show.jl
Original file line number Diff line number Diff line change
Expand Up @@ -755,7 +755,7 @@ end

# PR 17117
# test print_array
let s = IOBuffer(Vector{UInt8}(), true, true)
let s = IOBuffer(Vector{UInt8}(), read=true, write=true)
Base.print_array(s, [1, 2, 3])
@test String(resize!(s.data, s.size)) == " 1\n 2\n 3"
end
Expand Down