Phenotype Microarray data analysis in BioPython

Everyone who does computational biology and has wrote at least one Python script probably knows about the BioPython library. I personally remember going “Oh!” some years ago when I gave up writing my own (horrible, terrible, clunky) GenBank file parser and discovered it. Since then it has been a central part of almost all small scripts I needed to write. Recent versions have become even more useful, with the inclusion of a very cool KEGG API wrapper, which has the side-effect of putting together two well-designed bioinformatics software together!

It is then with great pleasure that I’m announcing the addition of the Bio.phenotype module to BioPython, starting from version 1.67. The module allows to parse and write the outputs of Phenotype Microarray experiments, as well as to run some simple analysis on the raw data. Even though I have published another software in the past to run the same analysis (plus some more), I thought that a simpler library would prove useful for many, and that having it as part of BioPython would make it more easily accessible. Moreover, from a software development perspective it is worth noting that BioPython is following very strict practices, which ensure that code is properly written, tested and maintained. This is all possible thanks to the great work of the BioPython community in general and of Peter Cock in particular.

Example PM well

An example well from a Phenotype Microarray experiment, showing the parameters estimated by the Bio.phenotype module (code is in the tutorial at the end of the post).

So, I really hope that this small library will prove useful to anyone with Phenotype Microarray CSV files collecting dust in their filesystem. To make it even easier I’ve posted a small tutorial here, which also includes some downstream analysis and plots that are not covered in the BioPython manual (the tutorial is also embedded at the end of this post).

Happy analysis!

Abstract “painting” with Fourier transforms

TL;DR: Fast Fourier Transforms can be used to make some pictures look a bit like abstract painting (here some scripts on top of ImageMagick).

One of the most recognized contemporary painters is Gerhard Richter; he basically has two distinct kind of paintings: “copies” of photographs and abstract paintings made in a procedural and somewhat mechanical way (have a look). Some years ago many of his paintings were hosted in the Strozzina museum in Florence; the very good guide challenged the crowd to understand the abstract paintings, whose impact on other artists was also shown in the very well-curated exhibition. I remember thinking that those paintings, by the “mechanical” way they were made, resembled the application of another mathematical procedure, called Fourier transform. For the non-technical crowd, I suggest reading a couple of articles to better understand what this transform is and most importantly how it affects everyday life: by a way of example, the mp3s that we listen to everyday are made through a series of Fourier transforms on the original sound wave, resulting in a much smaller file, which we can even stream on the fly over the internet. Its inverse is also used to interpret x-ray crystallography, which brought us the discovery of the DNA helical structure, above other things. To better understand how it works, you can have a look at this very nice article, or even play with it in your browser with this neat Flash (yuk) page.

After a couple of years I got involved in a discussion about Richter’s paintings and thought that would have been nice to see if it was possible to produce similar paintings using a somewhat automatic and mechanical procedure. Luckily, Fourier transforms are also used to produce JPEGs (the equivalent of mp3s for picture files), so bending this transform could have produced interesting results. The difference would have been that Richter uses is sensibility and experience to direct the process, while this other process would have been totally automatic, but also unpredictable to a certain extent.

After some research I’ve found a nice article that explains how ImageMagick, the popular Unix command-line tool for image manipulations, uses Fourier transforms and its inverse to manipulate pictures, showing for instance some practical examples on how to extract the phase and magnitude component from a picture and combining them back to the original file. What would have happened when thinkering a bit with this commands? I’ve done some experiments, and crafted a few bash scripts to make it a bit easier.

Reverse transform using only the image magnitude

The first experiment results are more similar to Richter paintings (not in terms of actual art of course!). Once the magnitude and phase components are extracted from a picture, much of the outlines of the image are stored in the phase component, while the colors are mainly found in the magnitude part (the fact that there’s no complete separation is a good thing, since it makes x-ray crystallography feasible). What happens when the inverse transformation is made using just the magnitude part? We should obtain a picture whose outlines are mostly gone, sprawled in an abstract way.

Magnitude transform

Magnitude transform (original picture on the left)

The results are even more surprising when a picture with regular patterns is used, demonstrating that part of the outlines are still retained somehow when doing the inverse transform.

Magnitude transform

Magnitude transform (original picture on the left)

Pretty abstract right? As you can see, using an image with a pattern generates more predictable results, which are still interesting. But we can do more…

Reverse transform with phase swap

Once we’ve extracted the magnitude and phase component we can separate them and do something stupid, like doing the reverse transform swapping the phase component of two images. Given the non complete separation of outlines and color, we can expect the resulting images to contain some of the tone and outline of the other. A bit like multiple expositions in photography, but with an abstract touch.

Swap transform (original pictures on top)

Swap transform (original pictures on top)

Again, using pictures with regular patterns results in interesting and slightly predictable results.

Swap transform (original pictures on top)

Swap transform (original pictures on top)

Sometimes, only one of the swaps looks interesting, usually due to the presence of patterns that somehow interact with each other.

Swap transform (original pics on tops)

Swap transform (original pictures on top)

These results look less abstract than the first trick, but look what happens when we use two pictures with clearly outlined patterns.

Swap transform (original pictures on top)

Swap transform (original pictures on top)

 Reverse transform combining two magnitudes

Swapping two images is quite interesting, but as we saw with the third example, sometimes only one of the swaps is worth keeping. What we could do is tampering with the magnitude component prior to the reverse transform, for instance by multiplying together the magnitude components of two different pictures (the original and another one). For this experiment I had no idea what to expect.

Multiply transform (original pictures to the left)

Multiply transform (original pictures on the left)

It’s really nice how the original image is still “there”, but gets contaminated by the other one. Again, using patterns produces predictable but cool results.

Multiply transform (original pictures on the left)

Multiply transform (original pictures on the left)

The fact that the second picture is mostly black produces a weird blurry effect, a bit like tampering with fresh paint on a canvas.

That’s it for now! Any addition or nice abstract result you get using your pictures is more than welcome. Let’s see if we can fool someone in hosting a Fourier transforms exhibition somewhere!

Through the Looking-Laptop

Laptops are weird, but also fragile, especially when they are overused and it comes the day when you know that they won’t boot again; the worst part (from an efficiency perspective) is that you can’t reuse most of its parts (apart from the HD and the RAM) leaving the rest (especially the monitor) unused and wasted. One thing you can do is transform the computer into something new, taking inspiration from all those forms of recycling art; since i was interested in recovering the monitor i stuck to that idea and turned my old, wasted laptop into a mirror, that someone with limited fantasy would call a geek-mirror.

Continue reading