python - build abi3 wheels in cibuildwheel

[K0lb3]

This PR makes it possible to build abi3 wheels within a cibuildwheel environment.
To not effect the performance of version specific builds the C code changes are all guarded within preprocessor conditions.

This PR also modifies the setup.py, making it:

• abi3 compatible
• python 2 compatible again
• simplifies the extension build code a bit

TODO:

• add type stubs (currently don't get included in wheel)
• fix abi3 setup.py build issue (C code works, setup.py still has some issues)

[K0lb3]

It might be worth discussing if the abi3 wheels should start at 3.6 as it's now, or 3.10, which would then allow using the Buffer interface in the abi3 wheels as well.

[eustas]

Awesome, thanks. Will review soon.

[anthrotype]

Thanks for this! I'd love to not have to worry about building brotli wheels every year that a new python minor version gets out :)

This looks good, except for the issue I raised in my previous comments about the unnamed package without its own folder, which must be addressed before merge.

[K0lb3]

Thanks for this! I'd love to not have to worry about building brotli wheels every year that a new python minor version gets out :)

The repo for building and publishing the wheels needs a PR as well to make this work in the end,
as pypi might otherwise block the upload due to already existing wheels of the same name.
Alternatively the cibuildwheel config can also be done within the pyproject.toml, and added to this PR.

e.g. like following which I use in one of my projects

[tool.cibuildwheel.linux]
archs = ["x86_64", "i686", "aarch64", "ppc64le", "s390x", "armv7l"]
build = "cp37-* pp3*"

[tool.cibuildwheel.macos]
archs = ["x86_64", "arm64"]
build = "cp37-macosx_x86_64 cp38-macosx_arm64 pp3*"

[tool.cibuildwheel.windows]
archs = ["AMD64", "x86", "ARM64"]
build = "cp37-win_amd64 cp37-win32 cp39-win_arm64 pp3*"

The archs are all written separately to ensure that cibuildwheel covers all possible ones instead of just the default ones.
pypy dosn't support abi wheels, so wheels for all version of it still have to be build.
Arm OSX earliest release python build is 3.8, therefore the version to be build for it is cp38.
Arm Windows starts at 3.9, so it's configured as cp39.

As brotli could also support 3.6 abi, the config above would have to be slightly changed, as armv7l linux support is only official for cp37...as far as I can recall.

Edit:
It's on purpose that the config has no universal(2) build, as pip will use the more specific one.
e.g. if pip finds universal2 and x86_64, then it will pick x86_64. Therefore there isn't really a point in universal builds imo.

[anthrotype]

Alternatively the cibuildwheel config can also be done within the pyproject.toml, and added to this PR.

I'd prefer to keep the wheel building in the separate repository as things are currently, though @eustas may have different views.

[K0lb3]

@eustas @anthrotype
I wish you a happy new year.
A bit late, but still better than an empty bump ping :P

[K0lb3]

@eustas @anthrotype
It's been another month......nearly 4 months, a third of a year, since your last reaction.

I had to basically "wipe" all of my commits and re-add them on a fresh branch as the master branch got multiple changes that totally broke my fork. As that took a bit of time, I don't plan to do this again.

[anthrotype]

LGTM

thank you very much for this, and apologies for the very late review (I was away on paternity leave until today).

I'll let @eustas merge this as I don't have direct commit rights on this repo.