Skip to content

🐞 Bug Bash wk 2🐞 Add: second lesson and images #120

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 19 commits into from
Closed
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix: edits to cleanup flow
  • Loading branch information
lwasser committed Jan 3, 2024
commit d9ed95d83024ee0fc56a18e250ecbf7ca212ecf3
124 changes: 67 additions & 57 deletions tutorials/1-installable-code.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ will have the bare minimum elements needed to be installable into a Python envir
1. Is it clear where to add commands? bash vs. Python console
Bash vs zsh is different
2. Does this lesson run as expected on windows and mac?
3. ADD: note about what makes something "package worthy", with a common misconception being that a package should be production-ready code that's valuable to a broad audience. this may not be a pervasive misconception in python, but a quick break-out with an explanation of what a package can consist of would be helpful.
:::

:::{figure-md} code-to-python-package
Expand All @@ -25,27 +26,34 @@ A basic installable package needs a few things: code, a [specific package file s

:::

:::{admonition} Learning Objectives
:::{admonition} About this lesson
:class: tip

In this lesson you will learn:

- How to make your code installable into any Python environment both locally and from GitHub
- How to create a basic `pyproject.toml` file to declare dependencies and metadata
- How to create a basic `pyproject.toml` file that includes package dependencies and metadata. This file is required to make your package installable.
- How to declare a [build backend](build_backends) which will be used to [build](build-package) and install your package
- How to install your package in editable mode for interactive development

To complete this lesson you will need a local Python (development)
environment. You are welcome to use any environment manager that you choose.
**What you need to complete this lesson**

To complete this lesson you will need a local Python
environment and shell on your computer.

You are welcome to use any Python environment manager that you choose.
If you are using Windows or are not familiar with Shell, you may want to [consult the Carpentries shell lesson.](https://swcarpentry.github.io/shell-novice/). Windows users will likely need to configure a tool for any Shell and git related steps.

* [If you need guidance creating a Python environment, review this lesson](extras/1-create-environment.md) which walks you through creating an environment using both `venv` and `conda`.
* If you aren't sure which environment manager to use and
you are a scientist, we suggest that you use `conda`, particularly if you are working with any sort of spatial data.
you are a scientist, we suggest that you use `conda`, particularly if you are working with spatial data.

In the upcoming lessons you will learn how to
**What comes next**

In the upcoming lessons you will learn how to:

* Add a README file to your package to support community use
* Add project metadata to your package to support PyPI publication
* Add additional project metadata to your package to support PyPI publication
* Publish your package to PyPI
:::

Expand Down Expand Up @@ -73,7 +81,7 @@ To make your Python code installable you need to create a specific directory str
- Some code.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doe my code need to do anything specific? for example, in R your package can be a single function, or a massive package that accomplishes a host of tasks. does Python have something similar?

it might also be helpful to provide some sample code so that anyone can run the tutorial with that code chunk.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's similar to R in this regard though I think it would be confusing to mention this.

I do love the sample code.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

friends - at the bottom of the lesson i have them actually create the module and do provide runnable code examples so they don't have to create their own code.

The top of the lesson explains what they are doing and the bottom is hands on - do the thing with everything they need.

there is also an example repo that (will have) the example replicated workflow that they can download / clone!! but it won't be done until we agree on all elements of the lesson (which we are super close to here i think!!)

i hope that helps.

@kierisi your question is an interesting one. Are you suggesting a small breakout perhaps that just says - you can have as many or as few functions, classes etc in a package? we could add something like that

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lwasser that would be really helpful! one thing I've seen a lot in the R community is confusion about what makes something "package worthy", with a common misconception being that a package should be production-ready code that's valuable to a broad audience. this may not be a pervasive misconception in python, but a quick break-out with an explanation of what a package can consist of would be helpful.

Copy link
Member Author

@lwasser lwasser Dec 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok great. noted. i'll try to add a small breakout about this to the text.

i think there are two types of "packages" here to talk about

  1. someone packaging up some code for the purposes of internal use. being installable in envts etc. vs
  2. something more broadly useful that you want others to use.

there's probably a spectrum around the above. pyOpenSci for instance wouldn't review any package - there are criteria around what is pyOpenSci "package worthy" . This is because we care about long term maintenance and use. in the latter case you'd then want tests and docs and general application. vs the former where something that just runs for the purposes of your work is good enough.

(and then everything in between 😆 )

- An `__init__.py` file in your code directory.

The directory structure you’ll create below looks like this:
The directory structure you’ll create in this lesson will look like this:

```bash
pyospackage/ # Your project directory
Expand All @@ -89,37 +97,25 @@ pyospackage/ # Your project directory

Notice a few things about the above layout:

1. Your package code lives within a `src/packagename` directory. We suggest that you use `src` directory as it ensures that you are running tests on the installed version of your code. However, you are welcome to instead use a [flat layout](https://www.pyopensci.org/python-package-guide/package-structure-code/python-package-structure.html#about-the-flat-python-package-layout) which does not have a src/ directory at the root. [Learn more here.](https://www.pyopensci.org/python-package-guide/package-structure-code/python-package-structure.html#the-src-layout-and-testing)
1. Your package code lives within a `src/packagename` directory. We suggest that you use `src` (short for **source code**) directory as it [ensures that you are running tests on the installed version of your code](https://www.pyopensci.org/python-package-guide/package-structure-code/python-package-structure.html#the-src-layout-and-testing). However, you are welcome to instead use a [flat layout](https://www.pyopensci.org/python-package-guide/package-structure-code/python-package-structure.html#about-the-flat-python-package-layout) which does not have a `src` directory at the root.
1. Within the `src` directory you have a package directory called `pyospackage`. Use the name of your package for that directory name.
1. In your package directory, you have an `__init__.py` file and all of your Python modules. You will learn more about the __init__.py file below.
1. The `pyproject.toml` file lives at the root directory of your package.
1. The name of the root directory for the package is **pyospackage** which is the name of the package. This is not a requirement but you will often see that the GitHub / GitLab repo and the root directory name are the same as the package name.

### What is an __init__.py file?

When a directory contains an `__init__.py` file, it can be imported directly into Python.
The `__init__.py` file tells Python that a directory
should be treated as a Python package. As such, a directory with an `__init__.py` file can be imported
directly into Python. The __init__.py file does not need
to contain any code in order for Python to recognize it; it can be empty.

For example, following the file structure example above which has an `__init__.py` file within it, you can run:

```python
import pyospackage
```

The `__init__.py` file tells Python that a directory should be treated
as a Python package.


:::{admonition} The **__init__**.py file
:class: tip

The __init__.py file does not need to contain any code, it can be
empty. Since Python 3.3 came out, you can install a package without an
`__init__.py` file. However, we suggest that you include empty __init__.py files in your
package structure as it allows you to customize your package’s user
experience.
:::


### What is a pyproject.toml file?

The **pyproject.toml** file is:
Expand All @@ -140,7 +136,8 @@ installable including:
:::{admonition} Why the pyproject.toml file is important
:class: tip

The `pyproject.toml` file replaces some of the functionality of both the setup.py file and setup.cfg files.
The `pyproject.toml` file replaces some of the functionality of both the
`setup.py` file and `setup.cfg` files.
If you try to pip install a package with no `pyproject.toml` you will get the following error:

```bash
Expand All @@ -154,18 +151,21 @@ Neither 'setup.py' nor 'pyproject.toml' found.

## Time to create your Python package!

Now that you understand the basics of the Python package directory structure, it's time to create a Python package! Below you will create a directory structure similar to the structure described above.
Now that you understand the basics of the Python package directory
structure, and associated key files (`__init__.py` and `pyproject.toml`),
it's time to create your Python package! Below you will create a directory
structure similar to the structure described above.

If you don’t wish to create each of the files and directories below, you can always [fork and clone and customize the pyOpenSci example package.](https://github.com/pyOpenSci/pyosPackage)
If you don’t wish to create each of the files and directories below, you
can always [fork and clone and customize the pyOpenSci example package.](https://github.com/pyOpenSci/pyosPackage)

## Step 1: Set Up the Package Directory Structure

Below you create the basic directory structure required
for your Python package. Note that there are instructions for creating the files and directories using shell. However you can also create files and directories in your preferred file directory tool (e.g. Finder on MAC or File Explorer on Windows) if you wish.
for your Python package. Note that there are instructions for creating the files and directories using shell. However you can also create files and directories in your preferred file directory tool (e.g. Finder on MAC or File Explorer on Windows or even a tool such as VSCode or Spyder) if you wish.

Create a new project directory for your package. Choose a
name for your package, preferably in lowercase and
without spaces (e.g., "pyospackage").
### Create your package's project directory structure
* Create a new project directory for your package. Choose a name for your package, preferably in lowercase and without spaces. For this tutorial we'll use `pyospackage`.

Inside the project directory:

Expand All @@ -176,12 +176,14 @@ Inside the project directory:
```bash
# Create a project directory in shell and a src directory within
mkdir -R pyospackage/src/pyospackage

# Change directory into pyospackage project dir
cd pyospackage

# View the current file structure
ls
```

### Add your `__init__.py` and `pyproject.toml` files
Next create two files:

- Inside the package directory, create a new file named `__init__.py` . This file ensures Python sees this directory as a package. You will use this file to customize how parts of your package are imported and to declare your package’s version in a future lesson.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well now you've labeled two directories as package. This is very ambiguous right now.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it. technically isn't

src/pyospackage the actual package directory (src is just a container for the package) whereas src/pyosppackage has the init.py file ?

would you agree? i've never thought about this but see how it could be super confusing. what is the package directory?

src/pyospackage or just the pyospackage dir?

so in shell we could do

Create the pyospackage project src and package directory like this:

mkdir -R pyospackage/src/pyospackage

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think I agree with you. And you may have figured this out with my other comments but src/pyospackage is the package directory because only its contents make it into the python package (distribution). The top level directory pyospackage is the project directory because it contains the package and more. A project could technically contain multiple packages (flit does this) but a package is contained by only one project.

Expand Down Expand Up @@ -212,7 +214,7 @@ If you don't have code already and are just learning how to
create a Python
package, then create an empty `add_numbers.py` file.

:::{admonition} Python modules and the __init__.py file
:::{admonition} Python modules and the `__init__.py` file
:class: tip

When you see the word module, we are referring to a `.py` file containing Python
Expand All @@ -226,6 +228,7 @@ modules.

:::

Your project directory should now look like this:
```
pyospackage/
└─ pyproject.toml
Expand All @@ -236,7 +239,7 @@ pyospackage/

```

## Step 3. Add code to your `add_numbers` module
## Step 3. Add code to your `add_numbers.py` module

If you are following along and making a Python package from scratch then you can add the code below to your `add_numbers.py` module. The function below adds two integers together and returns the result. Notice that the code below has a few features that we will review in future tutorials:

Expand Down Expand Up @@ -282,7 +285,7 @@ are welcome to copy the file we have in our [example pyospackage GitHub reposito
:::{admonition} Brief overview of the TOML file
:class: tip

The TOML format consists of tables and variables. Tables are sections of information denoted by square brackets:
[The TOML format](https://toml.io/en/) consists of tables and variables. Tables are sections of information denoted by square brackets:

`[this-is-a-table]`.

Expand Down Expand Up @@ -317,7 +320,7 @@ requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "pyospackage_gh_user_name" # rename this if you plan to publish to test PyPI
name = "pyospackage" # rename this if you plan to publish to test PyPI
# Here you add the package version manually.
# You will learn how to setup dynamic versioning in a followup tutorial.
version = "1.1"
Expand Down Expand Up @@ -349,7 +352,7 @@ You are now ready to install (and build) your Python package!

Let’s try it out.

- First open bash and `cd` into your package directory
- First open your preferred shell (Windows users may be using something like gitbash) and `cd` into your project directory
- Activate the Python environment that you wish to use. If you need help with working with virtual environments [check out this lesson](extras/1-create-environment.md).
- Finally run `python -m pip install -e .`

Expand All @@ -358,11 +361,13 @@ Let’s try it out.
# Below we use conda but you can do the same thing with venv!
> conda activate pyosdev
(pyosdev)
>> conda info
> conda info
active environment : pyosdev
active env location : /Users/your-path/mambaforge/envs/pyosdev
# Install the package
>> python -m pip install -e .
# Cd into your project directory
> cd pyospackage
# Install your package
> python -m pip install -e .

Obtaining file:///Users/leahawasser/Documents/GitHub/pyos/pyosPackage
Installing build dependencies ... done
Expand All @@ -382,7 +387,7 @@ Let's break down `pip install -e .`
`pip install -e .` installs your package into the current active
Python environment in **editable mode** (`-e`). Installing your package in
editable mode, allows you to work on your code and then test the updates
interactively in your favorite Python interface. One important caveat of editable mode is that every time you update your code, you may need to restart your Python kernel.
interactively in your favorite Python interface. One important caveat of editable mode is that every time you update your code, you may need to restart Python.

If you wish to install the package regularly (not in editable
mode) you can use:
Expand Down Expand Up @@ -432,7 +437,7 @@ pyosPackage 0.1.0 /Users/yourusername/path/here/pyosP

## 6. Test out your new package

After installing your package, type “python” at the command prompt to start
After installing your package, type “python” at the command prompt in your chosen terminal to start
a Python session in your active Python environment.

You can now import your package and access the `add_num` function.
Expand All @@ -446,20 +451,6 @@ Type "help", "copyright", "credits" or "license" for more information.
3
```

## Congratulations! You created your first Python package

You did it! You have now created a Python package that you can install into any Python environment. While there is still more to do if you want to publish your package, you have completed the first major step.

In the upcoming lessons you will:

* Add a [README file](2-add-readme.md) and [LICENSE ](4-add-license-file.md) to your package
* [Add more metadata to your `pyproject.toml`](5-pyproject-toml.md) file to support PyPI publication.
* [Learn how to build your your package distribution](6-publish-pypi.md) files (**sdist** and **wheel**) and publish to **test PyPI**.
* Finally you will learn how to publish to **conda-forge** from **PyPI**.

If you have a package that is ready for the mainstream user then
you can also publish your package on PyPI.


:::{admonition} Installing packages from GitHub

Expand All @@ -469,4 +460,23 @@ always install packages directly from GitHub using the syntax:
```bash
pip install git+https://github.com/user/repo.git@branch_or_tag
```

To make your package github installable, you can simply:

1. Create a new GitHub repo
2. Push the contents of the project directory that you created above, to GitHub
3. Finally install the package from GitHub using the command above.
:::

## Congratulations! You created your first Python package

You did it! You have now created a Python package that you can install
into any Python environment. While there is still more to do if you want
to publish your package, you have completed the first major step.

In the upcoming lessons you will:

* Add a [README file](2-add-readme.md) and [LICENSE](4-add-license-file.md) to your package
* [Add more metadata to your `pyproject.toml`](5-pyproject-toml.md) file to support PyPI publication.
* [Learn how to build your package distribution](6-publish-pypi.md) files (**sdist** and **wheel**) and publish to **test PyPI**.
* Finally you will learn how to publish to **conda-forge** from **PyPI**.