Paddle: система сборки для языка Pyhon

Paddle is a young, extensible, and IDE-friendly build system for Python. It provides a declarative way for managing project dependencies, configuring the execution environment, running tasks, and much more. You can find more information about Paddle on its GitHub page, and even download the latest release in the form of a PyCharm plugin on JetBrains Marketplace.

The project was originally started as an initiative of the Grazie team at JetBrains. The team develops an ML-based writing assistance platform (with a number of related plugins and instruments), and they needed a lightweight build system to manage huge mono-repositories with different ML models in Python. Paddle was designed as a prototype of a general-purpose build system with support of the Python language, so that it is able to run different scripts, tests, linters, etc. Paddle is written in Kotlin, and that is why it can be easily integrated into any IntelliJ-based IDE (PyCharm for Python, specifically).

Since then, the codebase of Paddle has been growing, and now there are many directions for the further development:

• Integration of Python-related CLI tools/instruments. Paddle already supports running and configuring a number of Python utilities, such as pylint, mypy, pytest, twine, and setuptools. We also want to support other linters (black, yapf, flake8, isort, bandit, autoflake, etc.), packaging tools (pex), publishing tools (jfrog), test frameworks (Unittest), and test environments (tox).

• Remote/containerized execution of Paddle tasks via SSH/Docker. Figure out a way to manage a virtual file system and run any Paddle task in the isolated environment.

• Kotlin/Python interoperability. Paddle is written in Kotlin, while it mostly works with Python projects. The goal here is to implement a way to declare custom tasks and extensions in Python directly in the user’s Paddle project instead of contributing plugins to the Paddle repository. See Gradle’s buildSrc for details.

• Windows support. Currently, Paddle is supported natively only on macOS/Linux, and could be also launched within the Docker container. For Windows, there are several challenging problems that need to be addressed.

• Remote caches. Implement a way to manage all cached Python dependencies not only on the local instance, but on the remote (e.g., company-wide) mirror.

Project lead: oleg.smirnov@jetbrains.com (telegram: @oesmirnov)

Test assignment: https://docs.google.com/document/d/1NpTjAUVTSL-omFbttODT8Kxw7N7zjMicpFKdR0ClN5E/edit