Moar docs

python/.pre-commit-config.yaml

@@ -45,8 +45,3 @@ repos:
     hooks:
       - id: pycln
         args: [ --config=python/pyproject.toml ]
-  - repo: https://github.com/python-poetry/poetry
-    rev: '1.2.0b1'
-    hooks:
-      - id: poetry-check
-      - id: poetry-lock

python/CONTRIBUTING.md

@@ -17,28 +17,39 @@
   - under the License.
   -->
 
-# Contributing to the Iceberg Python libary
+# Contributing to the Iceberg Python library
+
+For the development, we use poetry for packing and dependency management. You can install this using:
+
+```bash
+pip install poetry
+```
+
+If you have an older version of pip and virtualenv you need to update these:
+```python
+pip install --upgrade virtualenv pip
+```
+
+To get started, you can run `make install`, which will install poetry and it will install all the dependencies of the Iceberg library. This will also install the development dependencies. If you don't want to do this, you need to install using `poetry install --no-dev`.
+
+If you want to install the library on the host, you can simply run `pip3 install -e .`. If you wish to use a virtual environment, you can run `poetry shell`. Poetry will open up a virtual environment with all the dependencies set.
 
 ## Linting
 
 We rely on `pre-commit` to apply autoformatting and linting:
 
 ```bash
-make install
 make lint
 ```
 
-By default, it only runs on the files known by git.
-
 Pre-commit will automatically fix the violations such as import orders, formatting etc. Pylint errors you need to fix yourself.
 
-In contrast to the name, it doesn't run on the git pre-commit hook by default. If this is something that you like, you can set this up by running `pre-commit install`.
+In contrast to the name suggest, it doesn't run the checks on the commit. If this is something that you like, you can set this up by running `pre-commit install`.
 
 ## Testing
 
-For Python, we use pytest in combination with coverage to maintain 90% code coverage
+For Python, we use pytest in combination with coverage to maintain 90% code coverage.
 
 ```bash
-make install
 make test
 ```

python/Makefile

@@ -16,13 +16,14 @@
 # under the License.
 
 install:
-	pip install -e ".[dev,arrow]"
+	pip install poetry
+	poetry install -E pyarrow
 
 lint:
-	pre-commit run --all-files
+	poetry run pre-commit run --all-files
 
 test:
-	coverage run --source=src/ -m pytest tests/
-	coverage report -m --fail-under=90
-	coverage html
-	coverage xml
+	poetry run coverage run --source=src/ -m pytest tests/
+	poetry run coverage report -m --fail-under=90
+	poetry run coverage html
+	poetry run coverage xml

python/README.md

@@ -30,7 +30,7 @@ cd iceberg/python
 pip install -e .
 ```
 
-Development is made easy using [Poetry](https://python-poetry.org/docs/#installation). 
+Development is made easy using [Poetry](https://python-poetry.org/docs/#installation).
 
 ## Development
 

python/dev/RELEASE.md

@@ -1,3 +1,22 @@
+<!--
+  - Licensed to the Apache Software Foundation (ASF) under one
+  - or more contributor license agreements.  See the NOTICE file
+  - distributed with this work for additional information
+  - regarding copyright ownership.  The ASF licenses this file
+  - to you under the Apache License, Version 2.0 (the
+  - "License"); you may not use this file except in compliance
+  - with the License.  You may obtain a copy of the License at
+  -
+  -   http://www.apache.org/licenses/LICENSE-2.0
+  -
+  - Unless required by applicable law or agreed to in writing,
+  - software distributed under the License is distributed on an
+  - "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  - KIND, either express or implied.  See the License for the
+  - specific language governing permissions and limitations
+  - under the License.
+  -->
+
 # How to release
 
 The guide to release the Python package.
@@ -114,7 +133,7 @@ Convenience binary artifacts are staged on pypi:
 
 https://pypi.org/project/apache-iceberg/$VERSION/
 
-And can be installed using: pip install apache-iceberg==$VERSION 
+And can be installed using: pip install apache-iceberg==$VERSION
 
 Please download, verify, and test.
 Please vote in the next 72 hours.
@@ -124,4 +143,4 @@ Please vote in the next 72 hours.
 EOF
 
 cat release-announcement-email.txt
-```
+```

python/pyproject.toml

@@ -39,13 +39,15 @@ packages = [
 
 [tool.poetry.dependencies]
 python = "^3.8"
-mmh3 = "3.0.0"
+mmh3 = "^3.0.0"
 
-pyarrow = { version = "8.0.0", optional = true }
+pyarrow = { version = "^8.0.0", optional = true }
 
 [tool.poetry.dev-dependencies]
-pytest = "7.1.2"
-pre-commit = "2.19.0"
+pytest = "^7.0.0"
+pytest-checkdocs = "^2.0.0"
+pre-commit = "^2.0.0"
+coverage = { version = "^6.0.0", extras = ["toml"] }
 
 [build-system]
 requires = ["poetry-core>=1.0.0"]