Asyncio in Python is a double edged sword, if you’re making lots of calls out to IO with very little actual
compute happening locally within the app it’s a godsend for performance. Honestly though, is that
something that you’re doing regularly? If you are great, keep doing your thing, if not then using vanilla
sync python is probably going to make your life easier.
I’ve inherited quite a bit of async code recently that doesn’t really need to be async. In a couple of
places I’ve seen code like this:
This is the first in an ongoing series of essential tools that I use day to day
as a python developer which I couldn’t be as productive without. This episode
is on factoryboy…
What do you use for fixtures?
Traditionally I built database fixtures one of two ways:
Manually crafted artisan SQL commands which populated required fields into
the database.
‘Live’ or environmental data that was dumped from the DB into a SQL file and
is imported into the test database at test run startup.
I normally would use method 1 for testing repository or service methods where
I needed a specific example which had some required properties, and method 2 for
general testing where I was more testing rough results rather than specific
outputs.
I think the dependencies you use for development should differ from the ones that
you use for testing. It’s quite easy to introduce dependencies in your code that are
based off of data or configuration that you have created outside of your codebase.
To get around this we can use an entirely separate set of test fixtures which are
created before each test run, configured, tested against and torn down when the
test run is over.
People calling expensive functions eagerly in the log event to fetch debugging data.
I see it so much at this point that I’m not even correcting it when I see it in the most of the
codebases I use. It seems too controversial of a thing to bring up, and honestly it’s not
worth spending some of your capital with your colleagues on. While I do love fstrings, there
are some great reasons outside of the above to not use them.
I’ve been battling bugs recently, this came up and it seemed like a fun exercise in test suite
hygiene. I take the view that test hygiene is everyones responsability regardles of the level
of the engineer, this is such a simple subject that everyone regardless of their skill level
can implement the ideas here.
Flaky tests are the bane of many a developers existence, even worse than this however is test
inter-dependencies. Let’s say we’ve got the following code, some function that marks a data
structure as being completed.
I’ve been using writing libraries and tooling for others recently and one of
the most useful tools in this has been gradual typing. In the libraries that
have been built we’ve fully typed them, allowing our users to make the decision
of opting into type checking or not depending upon their needs.
As we don’t know what objects are users are going to be passing into our
consuming functions we need a way of defining what properties our objects need
ahead of time without knowing what the object will be. This is static duck
typing; or to use it’s correct name as per the PEP, structural subtyping.
This is the first in an ongoing series of essential tools that I use day to day
as a python developer which I couldn’t be as productive without. First episode
is on pytest.
What is pytest?
This should be pretty obvious by the name, it’s a testing framework for Python.
Unlike the standard libraries unittest it doesn’t wholesale inherit ideas from
other languages, it reads, looks and works like you’d expect in Python. In my
view, there’s 3 main reasons to use pytest:
I’ve spent the past couple of weeks using line coverage to search out weakly
tested sections of code in a large codebase I’m working on. While this has been
quick at identifying areas that have room for improvement I think it’s worth
highlighting that line coverage isn’t always the best indicator of quality of
your tests.
Take this highly contrived example…
Hypothetical big bank requirement
We’re a new developer and we’ve been tasked with turning a new product
requirement on a mortgage service into reality. We’re not actually checking
whether a mortgage is deemed as affordable by some regulatory checks.
I’ve been involved in quite a few post mortems in my time, a running them
across almost all of them was that they weren’t down to a single cause but a
series of failures that combined to result in a damaging event. The most recent
post mortem was a data synchronisation service that uses an unstable API. It was
built by one team and inherited by another, errors have grown to be expected and
it’s mostly worked. Concerns have been raised but a working system is not often
prioritised. The dissociation between the engineers who built the code and
those now running it has led to a certain acceptance that some error conditions
can be ignored.
I love decorators. Python is a pretty concise language but all too often you
can end up with cluttered code that is full of monitoring and debugging logic.
Let’s take some pretty typical code and see if we can improve it with some
decorators.
This post uses timing as a contrived example, in the real world you’ll
probably just want to using a pre-existing timing decorator and be done with
it. Realistically you’ll find decorators useful for things like
authentication, metrics, logging, tracing and really anything where you want
to do something before or after a callable.
I love Varnish. As far as the world of HTTP goes, in environments where you can cache it’s one of the easiest quick wins there is. However this does somewhat come at a price to debugging. Often you’ll sit back and your black box of magic doesn’t quite produce the result you were expect from it.
Normally I end up adding a sleuth of headers cache status headers to help identify where things are going wrong. The first and most useful of these to add is X-Grace to show if Varnish thinks it has served a graced object. Graced objects are those that are outside of their TTL but are delivered anyway as no backend was available to service the request.