BFD-4351: Python CI improvements

[np-complete0]

JIRA Ticket:
BFD-4351

What Does This PR Do?

This PR centralizes Python dependency and tooling configuration by introducing a uv workspace in apps. It consolidates Python version pinning, dev tools, and lockfile management, and updates CI to run checks at the workspace level. It also removes redundant per-project .python-version files.

What Should Reviewers Watch For?

If you're reviewing this PR, please check for these things in particular:

• Verify the non-workspace ops project job behaves as expected and didn’t accidentally get included/excluded.
• Verify requires python in pyproject.toml and critical ops projects are compatible.
• Run the quick local commands to sanity-check the workspace:

Usage

From the apps directory:

Install all dependencies

uv sync --all-groups

Install dependencies for a specific project

uv sync --package bfd-pipeline-idr

Run commands

# Run tests
uv run pytest

# Run tests for a specific project
uv run pytest bfd-pipeline/bfd-pipeline-idr

# Check linting
uv run ruff check

# Format code
uv run ruff format --check

# Check types
uv run pyright

Add a dependency to a project

# Navigate to the project directory
cd bfd-pipeline/bfd-pipeline-idr

# Add a dependency
uv add <package-name>

Add a dev dependency

# Add to workspace-level dev dependencies
uv add --dev <package-name>

# Add to a specific project's dev dependencies
cd bfd-pipeline/bfd-pipeline-idr
uv add --dev <package-name>

Configuration

Python Version

The Python version (3.13) is specified in:

• Repository root .python-version
• apps/pyproject.toml (requires-python field)

Tool Configuration

Tool configurations (ruff, pyright) are centralized in apps/pyproject.toml under the [tool.*] sections. Individual projects can override these settings if needed.

Dependency Groups

• Common dev dependencies (ruff, pyright, pytest) are defined at the workspace level
• Project-specific dependencies are defined in each project's pyproject.toml
• Optional dependency groups (like lambda-run-locust for locust_tests) are project specific

CI/CD

The CI workflow (.github/workflows/ci-python.yml) runs workspace level checks:

1. Linting with ruff check
2. Formatting check with ruff format --check
3. Type checking with pyright
4. Tests with pytest

Python and tool versions are pinned in the CI workflow to ensure consistency.

Migrating from Individual Projects

Previously, each Python project had its own:

• .python-version file
• uv.lock file
• Tool configurations (ruff, pyright)

With the workspace setup:

• Single .python-version at repository root
• Single uv.lock at workspace root (apps/uv.lock)
• Unified tool configurations
• Simplified dependency management
• Faster CI/CD (single install step)

Troubleshooting

Lock file out of sync

If you see errors about the lock file being out of sync, run:

uv lock

Virtual environment issues

If you encounter virtual environment issues, remove and recreate:

rm -rf .venv
uv sync --all-groups

Project not found

Make sure the project is listed in the [tool.uv.workspace] section of apps/pyproject.toml.

---

What Security Implications Does This PR Have?

Please indicate if this PR does any of the following:

• Adds any new software dependencies

• No

• Modifies any security controls

• No

• Adds new transmission or storage of data

• No

• Any other changes that could possibly affect security?

• No

• I have considered the above security implications as it relates to this PR. (If one or more of the above apply, it cannot be merged without the ISSO or team security engineer's (@sb-benohe) approval.)

• I have created tests to sufficiently ensure the reliability of my code, if applicable. If this is a modification to an existing piece of code, I have audited the associated tests to ensure everything works as expected.

Validation

Have you fully verified and tested these changes? Is the acceptance criteria met? Please provide reproducible testing instructions, code snippets, or screenshots as applicable.

[malessi]

I haven't taken too much of a look at everything else quite yet, but from my cursory glance it looks like this is going in the right direction! The pyproject.toml should get moved to the root, first.

done

[np-complete0]

apps/pyproject.toml and ops/pyproject.toml have been merged into /pyproject.toml
@aschey-forpeople @malessi

[bfd-sast]

Quality Gate passed for 'bfd-parent'

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube