Using Python on macOS Sonama

I wanted to use Simon Willison's shot-scraper tool. This made me understand how to set up a Python dev environment.

First, I upgrade Python:

~ % which python3
/opt/homebrew/bin/python3
~ % python3 --version
Python 3.12.2
~ % brew upgrade python3
~ % python3 --version
Python 3.12.3

Then I attempt to install shot-scraper:

screenshots % pip install shot-scraper
zsh: command not found: pip

I try pip3 install shot-scraper. This works, but doesn't install the package:

screenshots % pip3 install shot-scraper
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try brew install
    xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a Python library that isn't in Homebrew,
    use a virtual environment:
    
    python3 -m venv path/to/venv
    source path/to/venv/bin/activate
    python3 -m pip install xyz
    
    If you wish to install a Python application that isn't in Homebrew,
    it may be easiest to use 'pipx install xyz', which will manage a
    virtual environment for you. You can install pipx with
    
    brew install pipx
    
    You may restore the old behavior of pip by passing
    the '--break-system-packages' flag to pip, or by adding
    'break-system-packages = true' to your pip.conf file. The latter
    will permanently disable this error.
    
    If you disable this error, we STRONGLY recommend that you additionally
    pass the '--user' flag to pip, or set 'user = true' in your pip.conf
    file. Failure to do this can result in a broken Homebrew installation.
    
    Read more about this behavior here: <https://peps.python.org/pep-0668/>

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

Claude gives me some options:

The error message you're getting is because of the way Homebrew manages and isolates Python packages to prevent conflicts with the system's Python installation.

Here are a few options to address this issue:

1. Use Homebrew to install Python packages

You can use Homebrew to install Python packages by running:

brew install <package_name>

Replace <package_name> with the package you want to install. This will install the package in a separate Homebrew-managed location, avoiding conflicts with the system's Python.

2. Create and use a virtual environment

Virtual environments are isolated Python environments that allow you to install packages without affecting your system's Python installation or other virtual environments. To create and use a virtual environment, follow these steps:

python3 -m venv /path/to/your/venv
source /path/to/your/venv/bin/activate
pip install <package_name>

Replace /path/to/your/venv with the desired location for your virtual environment, and <package_name> with the package you want to install. After activating the virtual environment, you can use pip to install packages within that isolated environment.

3. Use pipx (recommended)

pipx is a tool that automatically creates virtual environments and installs packages in an isolated location. It's a convenient way to manage Python packages without interfering with your system's Python installation. Install pipx first:

brew install pipx
pipx ensurepath

Then, you can install packages with pipx:

pipx install <package_name>

This will create a virtual environment for the package and add the package's executable to your PATH, making it easy to run.

By following one of these approaches, you can install Python packages on your macOS system without running into conflicts with Homebrew's managed Python installation.

I've going to try pipx, although I'm not entirely sure what it is:

pipx install shot-scraper

Whilst it's installing, I look for more info about pipx. Its README says:

pipx is a tool to help you install and run end-user applications written in Python. It's roughly similar to macOS's brew, JavaScript's npx, and Linux's apt.

I know what npx is so I now have a good mental model. Jacob adds to my understanding:

Why? pipx lets me install Python-based CLI stuff (youtube-dl, awscli, doc2dash, etc.) without those projects’ dependencies messing up my global Python.

shot-scraper has installed successfully:

  installed package shot-scraper 1.4, installed using Python 3.12.3
  These apps are now globally available
    - shot-scraper
done! ✨ 🌟 ✨