Python

Pyenv

Latest Version

In fish, you don't need the \ in dev\|rc.

python-build --definitions \
    | grep "^3" \
    | grep -v "dev\|rc" \
    | tail -n 1

#+RESULTS:

3.11.0a2

Using a src directory

Packaging a python library (Ionel Cristian Mărieș)
Testing & Packaging (Hynek Schlawack)

I am currently a fan of the src directory. The main reasons for this are:

Ionel's first point about import parity; it is important that all your top-level repo cruft isn't on your sys.path during local development.
A directory named src is explicit and says what it is.
Different python projects can have a more similar top-level directory layout and a more similar setup.cfg.

Packaging

Kinds of Packages

From: https://bernat.tech/posts/pep-517-and-python-packaging/

wheel (.whl)

The project is built and then zipped up. Building usually means executing setuptools, which in turn may call build scripts in the project.

If the package includes native extensions, they are complied. This means a separate wheel must be created and published for each computer architecture.

The zip includes the source code, the .dist_info directory, and potentiall compiled native extensions. During a pip install, pip only needs to extract the zip in the right location.

Source Tree (git)

Every file in the project's VC repository.

A zip of the project source code. It may not include all files in the repository, like the .git/, .github/, Jenkinsfile, and tests (but I think it can, right?). It must at least include the Python source code and files required to build the project (setup.cfg, etc).

Does not include a .dist_info/ or compiled extensions. During a pip install, pip needs to build the package, which usually means executing setuptools, which in turn may call build scripts.

The installing user must:

Have an appropriate version of setuptools
Have the tools required by the project build scripts

Package summary info

sys.path

As explained in the docs, the first element of sys.path is often the empty string.

… the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first.

import sys; return sys.path

site

Print a summary including sys.path and some other stuff.

python -m site

Interactively, in a python interpreter, site_script() prints some nice stuff, but it causes the interpreter to exit! Ok for org-mode, not great for an interactive python session.

import site
site._script()

Class attributes

Define a class:

class Pizza(object):
    def __init__(self, size):
        self.size = size
    def get_size(self):
        return self.size

Create an instance:

p = Pizza(8)

From: https://blog.ionelmc.ro/2015/02/09/understanding-python-metaclasses/#object-attribute-lookup

Now, calling p.get_size() or p.size roughly equates to:

Call the type slot for Class.__getattribute__(attribute). The default does this:
- Does Pizza.__dict__ have a size (or get_size) that has a __get__ method and is a data descriptor? (size: no, get_size: no). Note: functions are not data descriptors. A data descriptor must have at least a __set__ OR a __delete__ attribute.
- Else, does p.__dict__ have a size (or get_size) item in it? (size: yes, get_size: no)
- Else, does Pizza.__dict__ have a size (or get_size) item that is not a data descriptor?

Pizza.__dict__

Pizza.__dict__["get_size"].__get__(p, Pizza)()

Require zip of github repo

You can get a tarball or zipball of a repo as documented here: https://developer.github.com/v3/repos/contents/#get-archive-link

In requirements.txt, you can do something like https://github.com/cfclrk/python-demo/zipball/master#egg=botocore

e.g. see: https://github.com/aws/aws-cli/blob/1973ca9709d41121b45c1a0d764ef6788028c708/requirements.txt#L6

Strings

Unicode

return len("😂")

#+RESULTS:

From: https://hsivonen.fi/string-length/

return len("🤦🏼‍♂️")

#+RESULTS:

Decode: convert bytes to UTF-8 string

A "byte string" is a byte literal, which is an immutable sequence of bytes (integers). A byte literal can be defined using ASCII characters up through 127. Byte values 128-256 can be specified using an escape sequence or hex codes (see: Bytes Objects).

For example, this works:

foo=b"foo"

But this does not:

bar=b"ОФИС"

  File "<stdin>", line 1
SyntaxError: bytes can only contain ASCII literal characters.

To get a Python 3 string from bytes, you must know what text encoding was used to create the bytes. Usually, it's ASCII:

foo = b"foo"
return foo.decode("ascii")

#+RESULTS:

foo

What's going on is:

foo is a byte literal (array of integers)
For each array item, decode it using the ASCII codec, then encode it with UTF-8
The result is a UTF-8 encoded string

Encode: convert UTF-8 string to bytes

return "foo".encode("ascii")

#+RESULTS:

b'foo'

bar="ОФИС"
return bar.encode("utf-8")

#+RESULTS:

b'\xd0\x9e\xd0\xa4\xd0\x98\xd0\xa1'

return  "😂".encode("ascii")

Prints:

>>> "😂".encode("ascii")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode
character '\U0001f602' in position 0: ordinal not
in range(128)

Random String

import random
import string
return "".join(random.choice(string.ascii_letters) for _ in range(8))

#+RESULTS:

knMyinMG

How I Set Up Python

Use pyenv to install python versions, and pyenv-virtualenv to create virtual environments per project.

First, install a couple versions of python.

pyenv install $LatestPython
pyenv install 3.6.8

I set pyenv global to the latest python I have installed. This is so that one-off python commands on the CLI use a tolerable version of python regardless of what directory the CLI is in.

I actually set pyenv global to a virtual environment based on the latest python I have installed. This way, whenever the global python environment gets too cluttered with pip packages, I just blow it away and recreate it.

Global virtual environment name:

echo $LatestPython | sed s/\\.//g

#+RESULTS:

Create a virtual environment for the global python, and set pyenv global to it.

pyenv virtualenv $LatestPython $GlobalVenv
pyenv global $GlobalVenv