Back to blog

Shipping something onto PyPI

· 4 min read
pythondjangopypiopen-source

A while back, I published semaphore-sms: a small API wrapper for Semaphore, a local SMS gateway here in the Philippines. That was my first time putting something on PyPI, and it was mostly just wrapping REST endpoints with a Python client. Nothing too complex, but it got me familiar with the process.

This time, I wanted to ship something with a bit more substance: django-managed-commands.

The problem

If you’ve worked on Django projects long enough, you’ve probably seen management commands that:

  • are supposed to be ran only once, but nothing actually enforces that
  • have no way to tell if they were already executed (especially painful in multi-tenant setups)
  • were never tested because nobody wrote unit tests for them (and the code somehow made it to main branch 🥀)

I’ve been in teams where somebody creates a management command, and later there’s zero traceability, no record of whether it was ran, when, or if it even succeeded. Sometimes people try sticking a call_command() inside an empty DB migration, and then it breaks later when the schema changes underneath it.

I’d seen a similar internal tool in a previous team and thought: why not make this a proper package and bring it into my current project?

What it does

django-managed-commands gives you a ManagedCommand base class that automatically handles execution tracking, database transactions, dry-run mode, and run-once enforcement. It also ships with a generator that scaffolds the command and its tests:

Terminal
$

Could a senior engineer write this in an hour? Likely, as the core logic isn’t groundbreaking.

Why I took the time to do it

The value for me was mainly so it’s more difficult for the team to screw up with management command execution, especially in our case where we’re dealing with multiple tenants in our instance, and we’re maintaining on-prem instances for customers. It’s all achieved by standardized test scaffolding, execution recording, the --dry-run flag.

It was also fun to go through the full process of packaging and publishing it properly.

Some takeaways

There is such a thing as Test PyPI

Test PyPI is a separate instance of PyPI that exists purely for testing. You can publish your package there, install it in a clean environment, and verify everything works before you push to the real thing.

If you’ve never shipped a package before, I’d strongly recommend publishing to Test PyPI first. It’s the same flow, zero risk.

Publishing makes you (try to) write better code

There’s something about knowing other people could install your package that makes you more careful. You think twice about developer experience, you actually write docstrings and a good README, you set up CI badges not just for vanity but because you want to signal that the thing is maintained. I found myself being extra mindful about edge cases and error messages in ways I might not be if this were just internal tooling.

File generation is an interesting rabbit hole

The create_managed_command generator in the package creates both the command file and a test file from templates. It’s a small feature, but writing it made me think about boilerplate generation more broadly. At some point I’d like to build a proper scaffolding tool for when I start new full-stack projects, something that sets up the directory structure, CI config, linting, the whole thing.

CI/CD for publishing is almost there

I set up a GitHub Action to automatically publish to PyPI on release creation, which was cool. It works great except for versioning. The version lives in pyproject.toml, and right now I have to bump it manually before creating the release. In the case where I forget, the action will currently naively try to publish, but PyPI rejects duplicate versions, so I get a failed deploy.

This is something I have yet to look for a workaround for, but at least it’s an issue just on my side and not for the ones who could use the package.

Wrapping up

The package itself is pretty straightforward, nothing special, but it was nice to scratch my own itch, and publish it while writing docs that a stranger could follow. It’s also pretty cool to be able to say “hey I published something on PyPI you might need in your project” ✨

If you’ve been thinking about publishing something, I’d say just go for it. It doesn’t have to be novel. Sometimes the point is the process itself, and getting first-hand experience on things that were always just a black box to you.