Skip to content

Commit 579b33c

Browse files
authored
getfile method and docs (#20)
* getfile method and docs * grammer * version bump
1 parent c9627ce commit 579b33c

File tree

5 files changed

+49
-4
lines changed

5 files changed

+49
-4
lines changed

CHANGELOG.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,17 @@ All notable changes to this project will be documented in this file.
44
The format is based on [Keep a Changelog](http://keepachangelog.com/)
55
and this project adheres to [Semantic Versioning](http://semver.org/).
66

7+
## [0.1.6] - 2018-01-31
8+
9+
### Added
10+
11+
- implemented new getfile method
12+
13+
### Changed
14+
15+
- Updated fs for more efficient directory walking
16+
- Relaxed boto requirement
17+
718
## [0.1.5] - 2017-10-21
819

920
### Added

docs/index.rst

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,6 @@ Amazon S3 cloud storage.
1313
As a PyFilesystem concrete class, S3FS allows you to work with S3 in the
1414
same as any other supported filesystem.
1515

16-
1716
Installing
1817
==========
1918

@@ -49,6 +48,28 @@ S3FS Constructor
4948
:members:
5049

5150

51+
Limitations
52+
===========
53+
54+
Amazon S3 isn't strictly speaking a *filesystem*, in that it contains
55+
files, but doesn't offer true *directories*. S3FS follows the convention
56+
of simulating directories by creating an object that ends in a forward
57+
slash. For instance, if you create a file called `"foo/bar"`, S3FS will
58+
create an S3 object for the file called `"foo/bar"` *and* an
59+
empty object called `"foo/"` which stores that fact that the `"foo"`
60+
directory exists.
61+
62+
If you create all your files and directories with S3FS, then you can
63+
forget about how things are stored under the hood. Everything will work
64+
as you expect. You *may* run in to problems if your data has been
65+
uploaded without the use of S3FS. For instance, if you create a
66+
`"foo/bar"` object without a `"foo/"` object. If this occurs, then S3FS
67+
may give errors about directories not existing, where you would expect
68+
them to be. The solution is to create an empty object for all
69+
directories and subdirectories. Fortunately most tools will do this for
70+
you, and it is probably only required of you upload your files manually.
71+
72+
5273
Authentication
5374
==============
5475

fs_s3fs/_s3fs.py

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -624,6 +624,19 @@ def getbytes(self, path):
624624
)
625625
return bytes_file.getvalue()
626626

627+
def getfile(self, path, file, chunk_size=None, **options):
628+
self.check()
629+
if self.strict:
630+
info = self.getinfo(path)
631+
if not info.is_file:
632+
raise errors.FileExpected(path)
633+
_path = self.validatepath(path)
634+
_key = self._path_to_key(_path)
635+
with s3errors(path):
636+
self.client.download_fileobj(
637+
self._bucket_name, _key, file
638+
)
639+
627640
def exists(self, path):
628641
self.check()
629642
_path = self.validatepath(path)

fs_s3fs/_version.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
__version__ = "0.1.5"
1+
__version__ = "0.1.6"

setup.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,8 +23,8 @@
2323
DESCRIPTION = f.read()
2424

2525
REQUIREMENTS = [
26-
"boto3~=1.4.0",
27-
"fs~=2.0.12",
26+
"boto3~=1.4",
27+
"fs~=2.0.18",
2828
"six~=1.10.0"
2929
]
3030

0 commit comments

Comments
 (0)