Doing the right work, in the right direction. Over a year ago, and after several years of study, I recently took the path less traveled. I stepped “down” from a Product Manager role to a Data Scientist role. I say “down,” because many people used that word when referring to…

For massive digital products, we need a different way to tackle a new set of problems that arise with scale. Many initially successful digital products, despite great intentions, easily devolve into a mess of seemingly impenetrable technical debt, impossible stakeholder requests and relentless stress for all involved. This deterioration is…

Emotion detection over 100 million tweets — I ran 100 million COVID-related tweets through my previously posted emotion-detecting neural network. I won’t try to explain the reasons behind the results, since this is an area rife with correlation, Twitter’s US-bias and a of course a very complex system. However, some very interesting patterns emerge! …

I came across a well-prepared dataset provided by Google, with 58 000 ‘carefully curated’ Reddit comments, labelled with one or more of 27 emotions, e.g. anger, confusion, love. Google had used this to train a BERT model, in which they had varying success in emotion detection depending on the type…

An ordered list of your biggest potential training gains, based on sports watch data from you and other athletes. — Running smart? As a runner always looking to improve my Personal Bests (PBs), I often wonder how I can optimise my time spent running to achieve the best gains whilst being as lazy as possible. Sometimes I run seven times a week, but don’t improve my speeds over a few months. Other…

Today’s dizzying number of options might just be making us miserable — I have been rather disappointed with Netflix lately. After it delisted a few of my favorite shows (ahem, Doctor Who) and I spent a few weeks struggling to find anything good to watch, I started wondering if the Netflix overlords had adjusted their strategy. …

The Movie DB (TMDB) provides an API for film data, the data which can be downloaded from here. I strove to find out whether, knowing only things I could know before a film was released, what the rating and revenue of the film would be. What parameters best predict a…